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General 


SECTION  1 
INTRODUCTION 


The  present  report  introduces  a  measurement  concept  for 
assessing  the  performance  of  an  Army  corps  participating  in 
a  joint  exercise  such  as  those  conducted  by  the  U.S. 
-Readiness  Command  (REDCOM) .  The  measurement  system  proposed 
is  a  dynamic  rather  than  static  system  which  can  be  tailored 
to  provide  feedback  relevant  to  the  unique  aspects  of 
different  corps  conducting  different  types  of  operations. 
The  measurement  concept  is  presented  in  a  preliminary  form 
and  represents  only  the  first  step  in  the  development  of  a 
comprehensive  measurement  and  development  program  at  the 
corps  level  in  the  Army. 

Background 

The  impetus  for  the  development  of  the  measurement 
concept  in  the  current  research  effort  comes  from  the 
growing  recognition  that  a  key  component  missing  from  many 
Army  training  systems  is  an  adequate  performance  measurement 
system.  A  recent  GAO  report  (1985)  evaluating  the  National 
Training  Center  (NTC)  suggests  that  many  units  training  at 
the  NTC  are  continuing  to  make  the  same  mistakes  made  by 
earlier  units.  The  authors  of  the  GAO  report  suggest  that  a 
key  factor  impeding  improvement  in  unit  performance  at  the 
NTC  is  the  lack  of  an  adequate  performance  measurement  and 
feedback  system.  The  final  report  of  the  1985  summer 
session  of  the  Army  Science  Board  (Training  and  Training 
Technology  -  Applications  for  AirLand  Battle  and  Future 
Concepts,  Peden  and  Barth,  1985)  identified  the  same 
problem.  The  authors  of  the  Army  Science  Board  report  note 
that  a  missing  link  degrading  the  effectiveness  of  many  Army 
training  and  development  programs  is  adequate  performance 
measurement.  This  problem  is  noted  as  particularly  acute  at 
higher  organizational  levels  where  measurement  of  either 
individual  or  organizational  performance  is  difficult. 

The  U.S.  Army  Research  Institute  has  undertaken  a 
number  of  research  efforts  designed  to  aid  in  development  of 
performance  measurement  systems  for  the  Army.  The  research 
to  develop  a  measurement  concept  for  assessing  corps-level 
performance  represents  one  facet  of  the  research  program  of 
the  Executive  Development  Research  Group  in  the  Manpower  and 
Personnel  Research  Laboratory. 
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Objective 


The  objective  of  the  current  research  effort  is  to 
develop  a  performance  measurement  concept  for  assessing  the 
performance  of  Army  corps.  A  secondary  objective  is  to 
present  a  plan  for  validating  the  measurement  concept. 


Scope 


As  noted  by  Bernardin  and  Beatty  (1984)  and  other 
experts  in  the  area  of  performance  measurement  (Cummings  and 
Schwab,  1973),  the  development  of  an  effective  performance 
measurement  system  begins  with  a  clear  understanding  of  the 
purpose  of  the  measurement  system.  Performance  measurement 
systems  may  be  used  to  generate  data  for  evaluation, 
training  and  development,  or  performance  management  and 
motivational  objectives.  The  content  and  format  of 
performance  measurement  systems  are  likely  to  vary  depending 
on  the  primary  purpose  for  which  the  system  is  constructed. 
Furthermore,  as  these  experts  note,  it  is  not  practical  to 
develop  a  single  performance  measurement  system  which  can  be 
applied  in  all  settings  to  provide  data  to  achieve  all 
purposes . 

The  performance  measurement  system  developed  in  the 
current  research  effort  is  designed  primarily  as  a  tool  to 
aid  in  the  development  of  corps  commanders  and  their  staffs. 
As  noted  in  FC  100-15,  Corps  Operations  (1984) ,  the 
development  of  the  principal  staff  at  echelons  such  as  Corps 
and  echelons  above  corps  (EAC)  is  primarily  the 
responsibility  of  the  commander  and  his  chief  of  staff.  To 
accomplish  this  task,  however,  the  commander  must  have  the 
capability  to  exercise  his  staff  in  the  execution  of  their 
wartime  missions  and  receive  feedback  that  will  enable  the 
commander  and  his  principal  staff  to  assess  the 
effectiveness  of  their  performance  and  identify  the  causes 
of  problems  which  may  occur  during  training. 

Corps-Level  Training  and  Development 

The  development  of  corps  commanders  and  training  of  the 
corps  staff  is  a  complex  task.  Realistic  corps-level 
training  is  difficult  to  conduct  simply  because  of  the 
number  of  people  and  geographical  area  required  to  exercise 
a  corps  in  the  field.  On  the  other  hand,  certain  critical 
corps-level  functions  can  be  realistically  exercised  if  one 
focuses  on  the  capability  of  the  corps  commander  and  his 
staff  to  plan  and  coordinate  the  execution  of  major  tactical 
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operations.  In  describing  the  mission  of  corps,  FM  100-5, 
Operations  (1986) ,  notes  that,  ’’Corps  plan  and  conduct 
major  operations  and  battles.  They  synchronize  tactical 
activities  including  the  maneuver  of  their  divisions,  the 
fires  of  their  artillery  units  and  supporting  aerial  forces, 
and  the  actions  of  their  combat  support  and  combat  service 
support  (CSS)  units."  (P.  185)  As  further  noted  in  FM  100-5, 
the  key  to  synchronization  on  the  battlefield  is  planning 
conducted  by  commanders  and  their  staff. 

Given  the  importance  of  the  planning  and  coordination 
functions  of  the  corps  commander  and  his  staff,  the 
performance  measures  developed  in  the  current  research 
effort  are  designed  to  provide  feedback  on  these  functions. 
The  performance  measurement  concept  is  designed  for 
implementation  in  the  second  phase  of  a  three-phase  training 
and  development  system.  The  three-phase  system,  suggested 
by  General  Paul  Gorman  (USA,  Ret.),  is  currently  in  the 
concept  development  stage.  As  described  below,  the  program 
would  provide  potential  corps  commanders  with  the 
opportunity  to  develop  and  practice  the  cognitive  skills 
required  for  corps-level  warfighting  and  provide  corps 
commanders  with  the  opportunity  to  exercise  and  assess  their 
staff's  ability  to  understand  and  execute  the  corps 
commanders'  intent  and  concept  of  operation. 

Phase  1: _ Development  of  General  Officers*  Frames  of 

Reference 

The  first  phase  or  segment  of  the  training  program  will 
be  designed  for  individual  development  of  cognitive 
warfighting  skills  among  general  officers.  This  segment  of 
the  program  requires  tne  development  of  a  computer-based 
system  with  databases,  models,  and  networking  capabilities. 
The  system  will  allow  asynchronous  communications  and 
sharing  of  information  among  participants  in  the  development 
program.  The  system  would  be  very  similar  to  that  discussed 
in  the  final  report  of  the  1985  Professional  Development  of 
Officers  Study  Group  (PDOS)  (1985).  The  bulk  of  the 
development  program  would  concentrate  on  individuals  or 
small  groups  of  general  officers  wargaming  the  planning  and 
execution  of  corps-level  operations  using  combat  models  and 
associated  data  bases.  Appropriate  feedback  systems  must  be 
developed  for  the  combat  models  keeping  in  mind  that  the 
objective  of  this  phase  of  the  program  is  to  provide  general 
officers  with  a  frame  of  reference  and  limited  experience  in 
planning  operations  with  the  degree  of  complexity  which 
exists  at  the  corps  level. 
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Phase  2:  Corps  Staff  Development 


The  second  phase  focuses  on  the  ability  of  the  corps 
staff  to  positively  affect  synchronization  on  the 
battlefield  through  their  understanding  and  execution  of  the 
commander's  intent  and  concept  of  operation.  This  phase  of 
the  training  and  development  program  represents  a  shift  in 
focus  from  upcoming  corps  commanders  to  existing  corps 
commanders  and  their  staffs  and  the  objective  of  this  phase 
is  also  quite  different.  This  portion  of  the  training 
program  is  currently  under  development  and  includes  a  family 
of  combat  simulation  models  such  as  Joint  Theater  Level 
Simulation  (JTLS)  and  Joint  Exercise  Support  System  (JESS) 
which  can  serve  as  exercise  drivers  in  corps  level  or  higher 
Command  Post  Exercises  (CPXs)  or  Command  Field  Exercises 
(CFXs) .  These  exercises  provide  the  opportunity  for  the 
corps  commander  and  his  staff  to  practice  the  planning  of 
major  tactical  operations  and  the  execution  of  coordination 
and  logistical  activities  required  during  wartime.  The 
exercises  are  typically  joint  exercises  which  provide 
opportunities  for  the  commander  and  his  staff  to  practice 
interfacing  with  other  services.  In  fact,  the  primary  focus 
of  the  exercises  under  development  at  REDCOM  is  the  joint 
services  interface. 

A  key  factor  currently  missing  from  these  large-scale 
exercises  is  an  adequate  performance  feedback  system  which 
provides  diagnostic  information  to  the  commander  and  staff. 
While  the  models  used  as  exercise  drivers  provide  combat 
outcome  feedback,  information  on  the  performance  of  various 
elements  of  the  corps  staff  which  contribute  to  combat 
outcomes  is  provided  only  on  an  ad-hoc,  subjective  basis. 
The  exercise  driver  models  will  need  to  be  supplemented  by 
additional  performance  measurement  tools  which  provide  the 
data  required  to  link  staff  actions  to  combat  outcomes  and 
provide  feedback  on  the  level  of  performance  of  various 
Svaff  elements.  The  development  of  a  measurement  concept 
for  such  a  performance  measurement  system  is  the  primary 
objective  of  the  current  research  effort. 

Phase  3:  Implementation  of  Operations  in  the  Field 

The  third  and  final  phase  of  the  training  and 
development  system  for  corps  commanders  and  their  staffs 
would  involve  the  use  of  corps-level  training  exercises 
without  troops  (TEWOTs) .  These  exercises  would  allow  the 
corps  staff  to  practice  implementation  of  the  their  plans  in 
a  field  environment  with  distributed  command  posts  (CPs) .  A 
corps-level  TEWOT  could  be  conducted  with  small  elements 
representing  units  down  to  battalion  spread  over  a 
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geographic  region  reflecting  the  typical  area  of  operation 
of  a  corps.  The  logistical ,  navigational,  and  communication 
problems  introduced  by  this  field  environment  would  provide 
a  more  reasonable  assessment  of  the  corps-staff's  ability  to 
plan  and  execute  operations  in  combat.  The  information 

gained  from  a  performance  measurement  system  in  this  type  of 
exercise  would  provide  feedback  focused  on  staff 

implementation/execution  of  the  commander's  plan  of 
operations  as  well  as  the  staff's  ability  to  communicate  and 
coordinate  in  a  field  environment. 

Performance  feedback  systems  will  be  reguired  in  all 
three  phases  of  the  corps  training  and  development  program. 
It  is  important  to  note,  however,  that  the  current  research 
effort  is  focused  on  the  development  of  a  performance 
measurement  concept  for  only  the  second  phase  of  the 
training  system. 

Organization  of  the  Report 

The  remaining  sections  of  the  report  describe:  (1)  the 

research  approach  followed  in  the  current  effort;  (2)  the 

performance  measurement  concept  developed  in  the  effort;  (3) 
an  application  of  one  aspect  of  the  performance  measurement 
concept  to  a  historical  military  operation;  and  (4)  a  plan 
for  validating  the  measurement  concept  within  the  context  of 
joint  exercises  conducted  at  REDCOM.  The  measurement 
concept  presented  in  the  report  is  complex  in  nature.  To 
facilitate  the  communication  of  this  concept,  certain 
material  has  been  placed  in  appendices  rather  than  included 
in  the  text. 

Section  2  of  the  report  describes  the  research  approach 
followed  in  developing  the  performance  measurement  concept. 
This  approach  was  based  on  a  seven  step  process  model  for 
the  development  of  an  organizational  performance  measurement 
system.  The  seven  steps  in  the  model  include: 

1.  Identification  of  the  organizational  purpose/ objectives 
of  the  organizational  element  of  interest 

2.  Identification  of  the  outcomes  and  products  produced  by 
the  organizational  element  that  are  related  to  the 
accomplishment  of  its  organizational  objectives 

3.  Development  of  a  performance  model  to  identify 
individual,  organizational,  and  environmental  factors 
impacting  on  the  quality  of  the  outcomes  and  products 
identified  in  (2)  above 
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4.  Development  of  criterion  measures  to  assess  quality  of 
the  outcomes/products  of  the  organizational  element 

5.  Development  of  measurement  procedures  to  assess 
performance  factors  identified  in  the  performance  model 
(diagnostic  measures) 

6.  Development  of  a  criterion  and  performance  measurement 
feedback  system 

7.  Validation  of  the  criterion  and  performance  measures 
and  assessment  of  the  utility  of  the  feedback  system 

The  seven  step  process  and  specific  research  activities  in 
the  current  study  are  both  presented  in  this  section.  The 
seven  step  model  described  in  Section  2  provides  the  basis 
for  the  research  approach  used  in  the  study  as  well  as  the 
basic  structure  for  the  organization  of  the  remainder  of  the 
report . 

Section  3  describes  the  findings  from  the  review  of 
Army  doctrine  and  interviews  with  military  SMEs  in  the  area 
of  corps-level  doctrine.  The  focus  of  this  section  is  on 
the  identification  of  the  organizational  purpose  and 
objectives  of  the  Army  Corps.  This  represents  the  first 
step  in  the  development  of  an  organizational  performance 
measurement  system. 

Section  4  of  the  report  presents  potential  criterion 
dimensions  and  an  information  processing  model  of  corps 
performance.  This  section  briefly  summarizes  the  review  of 
published  literature  on  several  existing  performance 
measurement  systems  which  are  described  in  more  detail  in 
Appendix  B.  The  identification  of  the  potential  outcome 
measurement  dimensions  and  development  of  the  performance 
models  are  the  second  and  third  steps  in  the  general  model 
for  developing  performance  measurement  systems. 

Section  5  of  the  report  provides  a  detailed  discussion 
of  the  concept  of  synchronization.  This  concept  is  the 
major  performance  construct  underlying  the  corps  performance 
measurement  system. 

The  conceptual  discussion  of  synchronization  in  Section 
5  is  immediately  followed  by  a  detailed  analysis  of  a 
synchronized  corps-level  operation  in  Section  6.  The 
military  operation,  the  Inchon  Landing  Operation  of  the 
Korean  War,  is  analyzed  using  synchronization  measurement 
tools  which  are  part  of  the  general  performance  measurement 
concept  developed  in  the  current  research  effort. 
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Section  7  of  the  report  presents  the  general 
performance  measurement  concept  which  is  the  primary  focus 
of  the  current  research  effort.  The  concept  is  discussed  in 
terms  of  a  measurement  strategy  as  well  as  the  major 
structural  components  in  the  measurement  concept. 

Section  8  presents  a  detailed  discussion  of  the  data 
collection,  analysis  and  synthesis  required  to  implement  the 
measurement  concept  described  in  Section  7.  The  data 
collection  plan  is  set  within  the  context  of  the  joint 
exercise  environment  at  REDCOM.  An  overview  of  the  REDCOM 
exercise  program  is  presented  in  Appendix  C  of  the  report. 

The  final  section  of  the  report,  Section  9,  presents  a 
general  discussion  of  validation  of  performance  measurement 
systems  as  well  as  a  specific  plan  for  validating  the  corps 
performance  measurement  concepts  within  the  REDCOM  context. 
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SECTION  2 
RESEARCH  APPROACH 


The  Concept  Development  Process 

The  primary  objective  in  the  current  research  effort 
was  the  development  of  a  measurement  concept  for  assessing 
the  performance  of  corps-level  organizations.  Therefore, 
the  research  approach  employed  in  the  study  was  primarily  a 
concept  development  process.  The  approach  was  based  on  the 
seven  steps,  described  in  Section  1,  for  developing  an 
organizational  performance  measurement  system.  The  process 
model  was  developed  through  earlier  research  by  the  first 
author  of  this  report. 

The  seven  steps  are  essentially  a  list  of  tasks  which 
must  be  completed  to  develop  an  operational  performance 
measurement  system.  While  the  development  of  the 
performance  measurement  concept  in  the  current  research 
effort  did  not  require  completion  of  all  seven  tasks,  the 
model  provided  a  framework  for  development  and  evaluation  of 
the  measurement  concept.  Figure  2-1  presents  a  schematic  of 
the  seven  step  process  and  the  tasks  completed  in  the 
current  research  effort  which  addressed  each  of  the  seven 
steps.  Each  of  the  seven  steps  in  the  general  model  and  the 
corresponding  research  tasks  completed  in  the  current  study 
are  described  in  more  detail  below. 

Step  1;  Identification  of  Organizational  Objectives 

The  first  step  in  the  general  model  for  developing 
organizational  performance  measurement  systems  is  to 
identify  the  purpose  and  objectives  of  the  organizational 
element  of  interest.  It  is  important  to  note  that 
identification  of  the  objectives  for  a  particular  element  of 
an  organization  usually  requires  collection  of  information 
from  outside  of  the  element  of  interest. 

The  identification  of  the  general  purpose  and 
organizational  objectives  for  Army  corps  was  accomplished 
through  three  research  tasks:  review  of  current  Army 
doctrine,  interviews  with  SMEs  working  on  corps-level 
doctrine  for  the  Army,  and  review  of  selected  historical 
examples  of  corps-level  operations  in  WWII  and  the  Korean 
War. 
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■  Doctrine  Review 

■  SME  Interview# 

•  Review  of  Historical  Examples 


a  Doctrine  Review 
•  REDCOM  Site  Visits 


e  Performance  Literature  Review 
e  Corps  SME  Interviews 
e  Brainstorming 


•  Warrior  Preparation  Center  SMEs 

•  Performance  System  Review 

•  REDCOM  Site  Visit 

a  Inchon  Landing  Example 
a  Brainstorming 

a  REDCOM  Discussions 
a  Brainstorming 
a  Inchon  Landing  Example 


a  Inchon  Application 
a  SME  Reviews 
•  Validation  Plan 


FIGURE  2-1.  Organizational  Performance 
Measurement  System:  Development  Process  Model 


2-2 


\ 


Step  2:  Identification  of  Organizational  Outcomes/Products 

Once  the  organizational  purpose  and  objectives  of  the 
element  or  unit  have  been  established,  the  next  step  is  to 
identify  potentially  measurable  outcomes  and/or  products 
which  contribute  to  achievement  of  the  organizational 
objectives.  A  critical  factor  in  successful  completion  of 
this  task  is  the  identification  of  outcomes  and  products 
that  are  directly  related  to  the  actions  of  the  element  of 
interest . 

In  the  current  research  effort,  the  identification  of 
the  products  or  outcomes  produced  by  Army  corps  was  a 
difficult  task.  The  decision  was  made  to  focus  on  the  corps 
commander  and  the  corps  staff  as  the  primary  organizational 
elements  of  interest.  Three  research  tasks  were  undertaken 
to  identify  the  products  or  outcomes  directly  associated 
with  the  corps  commander  and  staff.  These  tasks  included 
the  review  of  current  Army  doctrine  related  to  corps  staff 
operations,  brainstorming  sessions  among  the  core  research 
team  and  selected  SMEs,  and  a  site  visit  to  identify 
potentially  measurable  outcomes  in  the  joint  exercises 
conducted  at  REDCOM. 

Step  3:  Development  of  a  Performance  Model 

This  step  involves  the  development  of  a  performance 
model  delineating  the  individual,  organizational,  and 
environmental  factors  impacting  on  the  performance  of  the 
organizational  element  of  interest.  Such  a  model  also 
includes  descriptions  of  the  relationships  between  these 
performance  factors  and  may  need  to  be  expanded  to  include 
the  relationship  of  the  products/outcomes  of  the 
organizational  element  of  interest  to  the  larger 
organizational  performance  objectives. 

The  performance  model  and  performance  factors 
identified  in  this  third  step  should  be  generic  in  nature. 
That  is,  the  model  should  be  applicable  to  the  performance 
of  similar  organizational  elements  in  a  variety  of  similar 
situations.  For  example,  while  the  performance  factors 
should  remain  stable,  the  measurement  procedures  developed 
to  assess  the  performance  of  a  corps  staff  may  differ  across 
settings.  That  is,  data  collection  opportunities  and 
situational  requirements  may  dictate  that  operationalization 
of  a  particular  performance  factor  through  measurement 
procedures  will  change  as  one  moves  from  a  CPX  to  an  FTX. 
This  is  one  reason  that  the  development  of  criterion  and 
performance  measures  are  treated  as  steps  separate  from  the 
development  of  the  performance  model  and  identification  of 
performance  factors. 

The  development  of  a  corps  performance  model  was  a  key 
step  in  the  current  research  effort.  As  will  be  described 
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later,  the  model  focuses  on  the  synchronization  construct 
underlying  AirLand  Battle  doctrine.  Three  tasks  in  the 
research  approach  contributed  to  the  development  of  the 
performance  model.  The  first  of  these  tasks  was  a  review  of 
the  literature  on  performance  measurement  systems  developed 
for  Army  units  and  headquarters  staffs.  The  second  task  was 
to  interview  SMEs  with  command  experience  at  the  corps-level 
or  above.  The  third  research  task  was  extensive 
brainstorming  sessions  held  by  the  research  team,  including 
the  presentation  and  critique  of  initial  models  of  corps 
performance  factors. 

Step  4:  Development  of  Criterion  Measures  and  Performance 
Measures 

The  fourth  step  in  the  general  model  for  developing  a 
performance  measurement  system  is  the  development  of  the 
actual  criterion  measures  which  can  be  used  to  evaluate  the 
quality  of  ithe  outcomes  or  products  produced  by  the 
organizational  element.  This  process  may  involve 
development  of  two  levels  of  criterion  measures  —  measures 
for  both  direct  and  indirect  outcomes  related  to  performance 
of  the  element  of  interest.  These  criterion  measures 
provide  data  as  to  how  well  the  element  performed.  They 
provide  answers  to  questions  related  to  what  happened  but 
not  why  a  particular  level  of  performance  occurred.  To 
answer  the  latter  question,  one  must  move  to  the  fifth  step 
in  the  development  of  the  performance  measurement  system  — 
development  of  measurement  procedures  to  assess  the 
performance  factors  identified  in  the  performance  model. 

Step  5:  Development  of  Measurement  Procedures 

The  development  of  measurement  procedures  to  accurately 
assess  factors  influencing  the  quality  of  outcomes  produced 
by  an  organizational  element  is  especially  critical  for  a 
performance  measurement  system  designed  for  training 
purposes.  The  data  provided  by  these  measures  forms  the 
basis  for  diagnostic  feedback  required  for  improving 
organizational  performance.  As  noted  above,  the  exact 
nature  of  the  measurement  or  data  collection  procedures  will 
vary  with  the  context  in  which  the  data  are  collected. 

While  the  current  study  did  not  require  the  development 
of  operational  criterion  and  performance  measures,  the 
concepts  for  such  measures  are  the  primary  products  of  the 
research  effort.  The  research  approach  used  in  the  current 
study  combined  the  fifth  and  sixth  steps  into  a  general 
measurement  procedure  development  task.  Five  research  tasks 
were  included  in  the  completion  of  this  aspect  the  study. 
The  first  of  these  tasks  was  the  literature  reviews  of  Army 
doctrine,  historical  examples  of  corps-level  operations,  and 
existing  performance  measurement  systems.  The  second  task 
was  to  interview  SMEs  who  had  worked  on  the  development  of 
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corps-level  performance  measures  for  the  Warrior  Preparation 
Center  (WPC) ,  a  joint  exercise  center  for  the  Army  and  Air 
Force.  The  third  task  was  a  second  site  visit  to  REDCOM  to 
conduct  interviews  and  receive  a  demonstration  of  the  Joint 
Exercise  Simulation  System.  The  purpose  of  this  visit  was 
to  gather  detailed  information  on  performance  data  which 
might  be  collected  during  REDCOM  exercises  involving  Army 
Corps.  The  fourth  research  task  was  a  series  of 
brainstorming  sessions  designed  to  synthesize  information 
collected  from  the  literature  reviews,  site  visits,  and  SME 
interviews.  The  final  research  task  was  a  preliminary 
application  of  some  of  the  measurement  concepts  to  the 
analysis  of  the  Inchon  Landing  Operation  conducted  by 
General  Douglas  MacArthur  during  the  Korean  War. 

Step  6:  Development  of  a  Feedback  System 

After  the  actual  data  collection/measurement  procedures 
have  been  established  for  an  operational  performance 
measurement  system,  attention  must  then  be  devoted  to  the 
development  of  a  feedback  system  for  providing  data  to  the 
interested  parties.  The  content  and  format  of  the  feedback 
will,  of  course,  be  idirectly  related  to  the  purpose  of  the 
performance  measurement  system.  When  the  primary  purpose  is 
training,  particular  attention  must  be  paid  to  presentation 
of  the  diagnostic  feedback  in  a  manner  which  is  not  likely 
to  create  defensive  reactions  among  members  of  the 
organizational  element.  It  is  best  to  keep  the  feedback  as 
objective  as  possible.  Data  should  be  presented  which 
allows  the  members  to  objectively  evaluate  the  effectiveness 
of  their  own  performance.  The  diagnostic  feedback  should  be 
used  as  a  vehicle  for  identifying  problems  and  potential 
solutions. 

No  operational  feedback  system  was  developed  in  the 
current  research  effort.  However,  the  discussion  of  the 
performance  measurement  concept  includes  the  topic  of 
providing  feedback  to  the  corps  commander  and  the  corps 
staff.  Three  research  tasks  addressed  the  issue  of  the 
feedback  mechanisms  required  in  a  measurement  system  for 
addressing  corps  performance.  These  tasks  included  the 
discussions  at  REDCOM,  the  research  team  brainstorming 
sessions,  and  the  application  of  the  synchronization 
measurement  concept  to  the  analysis  of  the  Inchon  Landing 
Operation. 

Step  7:  Validation  of  the  Performance  Measurement  System 

The  final  step  in  the  development  of  a  performance 
measurement  system  is  the  validation  of  the  criterion  and 
performance  measures  and  assessment  of  the  utility  of  the 
feedback  system.  Ideally,  the  criterion  and  performance 
measures  would  be  validated  by  collecting  data  over  time  and 
across  different  organizational  elements.  These  data  would 
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be  correlated  to  Independent  performance  indicators  as  well 
as  examined  for  internal  consistency  in  relationships  among 
performance  factors  and  criterion  measures  contained  within 
the  performance  measurement  system.  Data,  in  the  form  of 
trainee  reaction  measures  regarding  utility  of  feedback,  are 
also  critical  in  the  evaluation  of  a  performance  measurement 
system  designed  for  training. 

Since  no  operational  performance  measures  were 
developed  in  the  current  research  effort,  no  empirical 
validation  tasks  were  included  in  the  research  approach. 
Initial  evaluation  of  the  feasibility  of  the  performance 
measurement  concepts  developed  in  the  study  was  accomplished 
through  the  application  of  the  concepts  to  the  analysis  of 
the  Inchon  Landing  Operation  and  through  reviews  by  SMEs 
familiar  with  corps  operations.  In  addition,  the  report 
contains  a  detailed  discussion  of  validation  strategies  that 
could  be  used  in  evaluating  a  measurement  system  based  on 
the  concepts  developed  in  the  current  effort. 

Execution  of  the  Research  Approach 

Figure  2-2  provides  a  schematic  overview  of  sequence  in 
which  specific  research  tasks  were  conducted  in  the  current 
study.  The  manner  in  which  each  of  the  major  research 
activities  identified  in  Figure  2-2  was  executed  is  briefly 
described  below. 

It  should  be  noted  that  throughout  the  entire  process  the 
contractor  and  the  Contracting  Officer's  Representative 
(COR)  engaged  in  a  series  of  intensive  brainstorming 
sessions  to  generate  and  evaluate  alternative  measurement 
concepts  and  potential  methodologies  for  implementing  the 
concepts . 

The  research  process  began  with  three  activities  which 
were  conducted  in  parallel.  These  activities  included  a 
review  of  several  bodies  of  literature,  brainstorming 
sessions  between  the  contractor,  the  COR,  and  various  SMEs, 
and  site  visits  and  interviews  with  individuals  involved  in 
the  various  aspects  of  doctrine  development  and  corps-level 
exercises. 
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Literature  Reviews 


Three  bodies  of  literature  were  reviewed  as  part  of  the 
measurement  development  process.  The  areas  on  which  the 
literature  reviews  focused  were  Army  doctrine,  recent 
historical  examples  of  corps-level  operations,  and  previous 
performance  measurement  systems  developed  for  the  Army.  The 
research  team  was  well  versed  in  the  area  of  performance 
measurement  theory  and  methodology.  The  primary  objective 
for  conducting  the  literature  reviews  was  to  ensure  that  the 
measurement  system  developed  in  the  current  research  effort 
would  have  a  sound  military  basis  and  provide  feedback  which 
was  meaningful  and  useful  to  corps  commanders  and  their 
principal  staff  officers. 

The  review  of  military  doctrine  had  two  foci:  First, 
the  latest  operational  doctrine  for  how  the  Army  plans  to 
fight  was  reviewed  in  detail.  The  foundation  for  this 
doctrine  is  the  AirLand  Battle  doctrine  presented  in  the 
1986  version  of  FM  100-5,  Operations.  In  addition  to  an 
intensive  review  of  FM  100-5,  the  authors  reviewed  a  number 
of  articles  discussing  the  history  and  issues  surrounding 
AirLand  Battle  doctrine  which  have  appeared  in  professional 
military  journals  such  as  Military  Review.  The  second  focus 
of  the  review  of  military  doctrine  was  directed  at  the 
identification  of  the  specific  roles,  objectives,  and  tasks 
to  be  accomplished  by  corps-level  organizations.  Documents 
reviewed  for  this  purpose  included  FC  101-55,  Corps  and 
Division  Command  and  Control  (1985);  FC  100-15,  Corps 
Operations  (1984);  and  FM  101-5,  Staff  Organization  and 
Operations  (1984) . 

The  review  of  historical  examples  of  corps-level 
operations  focused  on  operations  conducted  during  WWII  and 
the  Korean  War.  The  historical  accounts  were  examined  to 
identify  the  nature  of  corps-level  operations,  the  nature  of 
the  planning  involved,  critical  factors  impacting  on  success 
or  failure  of  the  operations,  etc.  As  the  measurement 
concept  developed,  the  review  of  corps-level  operations 
centered  on  operations  which  exemplified  synchronization, 
one  of  the  four  tenets  underlying  AirLand  Battle  doctrine. 
The  Inchon  Landing  conducted  by  General  Douglas  MacArthur 
during  the  Korean  War  was  studied  in  extensive  detail  and 
used  as  a  demonstration  of  the  application  of  the 
measurement  methodology  for  assessing  synchronization. 

The  review  of  existing  measurement  systems  focused  on 
previous  attempts  to  assess  the  performance  of  commanders 
and  their  staffs.  The  information  collected  during  the 
review  was  used  to  assess  the  viability  of  the  measurement 


concepts  developed  in  the  current  effort.  The  review  was 
also  used  to  identify  potential  problems  which  the  current 
system  needed  to  avoid  or  solve. 

Subiect-Matter  Expert  Interviews 

Concomitant  with  the  literature  reviews,  the  research 
team  met  on  separate  occasions  with  two  subject-matter 
experts  (SMEs)  with  command  experience  at  the  corps  level  or 
above.  The  interviews  were  highly  interactive,  semi- 
structured  and  ended  in  brainstorming  sessions.  One  of  the 
interviews  lasted  approximately  1/2  day  while  the  other 
lasted  for  approximately  two  days.  The  SMEs  met  with  a 
group  of  approximately  four  to  six  researchers.  The 
interviews  focused  on  three  topics:  the  critical  tasks  and 
objectives  of  corps-level  organizations,  methods  to  assess 
corps  performance,  and  methods  for  corps-level  training. 
Both  interviews  were  tape  recorded. 

Site  Visits 

In  addition  to  the  two  SMEs  with  command  experience  at 
the  corps  level  or  above,  the  research  team  made  a  number  of 
site  visits  to  interview  various  personnel  involved  in  the 
development  of  corps-level  doctrine  or  the  development  and 
implementation  of  corps-level  exercises.  The  doctrinal 
experts  were  interviewed  at  the  Combined  Arms  Center  (CAC) 
at  Ft.  Leavenworth,  KS.  and  at  TRADOC  Headquarters  at  Ft. 
Monroe,  Va.  The  primary  focus  of  these  interviews  was  on 
the  current  status  of  doctrine  related  to  corps  operations 
and  mission  essential  tasks  at  the  corps  level.  The  primary 
finding  from  the  interviews  with  doctrinal  experts  was  that 
corps-level  doctrine  is  in  a  state  of  transition. 

The  interviews  with  individuals  involved  in  the 
development  and  implementation  of  corps-level  exercises  were 
conducted  on  two  visits  to  REDCOM.  The  first  visit  was 
exploratory  in  nature  and  served  to  focus  the  research 
effort  on  development  of  a  performance  measurement  system 
for  implementation  in  the  joint  exercises  conducted  at 
REDCOM.  The  second  set  of  interviews  were  structured  with  a 
focus  on  collection  of  specific  information  about  the  nature 
of  the  Bold  Venture  87  and  later  exercises  to  be  conducted 
by  REDCOM.  A  copy  of  the  questions  addressed  during  the 
second  REDCOM  visit  is  included  in  Appendix  A.  The  research 
team  interviewed  individuals  involved  with  the  development, 
control,  and  evaluation  of  the  Bold  Venture  87  exercise. 
All  of  the  questions  contained  i~>  Appendix  A  were  answered 
during  the  interviews. 
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Input  from  Researchers  from  the  Warrior  Preparation  Center 


Besides  REDCOM,  the  only  location  at  which  joint 
exercises  are  conducted  using  computer  models  as  exercise 
drivers  is  at  the  Warrior  Preparation  Center  in  Europe.  The 
research  team  for  the  current  effort  met  with  Army  and  Air 
Force  research  psychologists  who  had  worked  on  performance 
.measurement  systems  for  the  Warrior  Preparation  Center.  The 
meetings  focused  on  identification  of  potential  problems  and 
solutions  related  to  performance  measurement  in  large-scale 
joint  exercises. 

Application  of  Synchronization  Measurement  Concept 

As  part  of  the  evaluation  of  the  feasibility  of  the 
measurement  concept  developed  in  the  current  effort,  the 
methodology  to  be  used  in  assessing  synchronization  of  a 
corps-level  operation  was  tested  using  detailed  accounts  of 
the  Inchon  Landing  Operation.  The  process  and  results  of 
this  application  are  described  in  detail  in  Section  6  of  the 
report . 

Review  of  the  Concept  bv  SMEs 

The  final  step  in  the  development  of  the  corps 
performance  measurement  concept  was  a  detailed  review  of  the 
concept  by  a  military  subject-matter  expert  currently 
working  on  the  development  of  an  Army-wide  training 
strategy. 

The  execution  of  the  research  tasks  described  above 
provided  the  research  team  with  a  wealth  of  information 
concerning  corps  operations.  The  analysis  and  synthesis  of 
the  information  produced  the  findings  described  in  Sections 
3  through  9  of  the  report.  The  next  section  of  the  report 
describes  the  findings  regarding  the  objectives  of  the  Army 
corps  as  identified  in  the  Army's  operational  doctrine. 
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SECTION  3 

IDENTIFICATION  OF  THE  PURPOSE  AND  OBJECTIVES 
OF  THE  ARMY  CORPS 


General 

The  Army  Corps  is  the  organizational  element  of  interest 
for  the  current  research  effort.  While  it  is  important  to 
recognize  that  each  Army  Corps  has  specific  objectives  and 
contingency  plans  which  make  it  unique,  one  can  identify  the 
general  role  which  doctrine  suggests  that  corps-level 
organizations  play  in  warfighting.  The  results  of  the 
research  tasks  conducted  to  identify  the  general 
organizational  purpose  and  objectives  of  Army  Corps  are 
presented  below. 

Review  of  Doctrine 

Before  discussing  the  specific  objectives  and  roles  of 
corps-level  organizations  in  warfighting,  it  is  important  to 
briefly  review  the  nature  of  the  Army's  operational  concepts 
for  the  modern  battlefield.  FM  100-5,  Operations  is  the 
doctrinal  manual  which  presents  the  concepts  which  underlie 
the  objectives  and  roles  of  organizational  elements  at  all 
levels  within  the  Army.  As  noted  in  the  preface  to  the  1986 
edition  of  FM  100-5: 

FM  100-5,  Operations,  is  the  Army's  keystone 
warfighting  manual.  It  explains  how  Army  forces 
plan  and  conduct  campaigns,  major  operations, 
battles,  and  engagements  in  conjunction  with  other 
services  and  allied  forces.  It  furnishes  the 
authoritative  foundation  for  subordinate  doctrine, 
force  design,  materiel  acquisition,  professional 
education,  and  individual  and  unit  training.  It 
applies  to  Army  forces  worldwide,  but  must  be 
adapted  to  the  specific  strategic  and  operational 
requirements  of  each  theater .  (p.  i) 

It  is  important  to  recognize  that  the  basic  operational 
concepts  underlying  Army  doctrine  as  presented  in  FM  100-5 
have  undergone  considerable  change  during  the  last  10 
years.  Prior  to  the  mid-1970s,  the  FM  100-5  presented  a 
doctrine  which  emphasized  both  firepower  and  maneuver.  In 
1976,  a  new  version  of  the  FM  100-5  was  published  which 
tended  to  focus  on  the  lethality  of  new  weapon  systems  and 
downplayed  the  role  of  maneuver  on  the  modern  battlefield. 
Instead,  the  manual  presented  a  doctrinal  approach  to 
warfighting  referred  to  as  the  "active  defense". 


3-1 


The  active  defense  doctrine  reflected  a  political  stance 
that  the  United  States  would  engage  in  war  only  for 
defensive  purposes  (with  a  primary  focus  on  heavy  forces  in 
Europe)  and  a  military  stance  that  lethality  of  current 
weapon  systems  required  fighting  from  well-prepared 
defensive  positions.  The  rationale  for  the  military  stance 
was  based  largely  upon  evaluations  of  the  1973  Middle  East 
War  and  various  computer  simulations  which  focused  attention 
on  firepower  as  the  primary  determinant  of  victory  on  the 
modern  battlefield.  As  noted  by  LTC  Paul  T.  DeVries,  US 
Army  (DeVries,  1983)  it  is  ironic  that  a  doctrine  which 
largely  neglects  maneuver  as  an  essential  element  of  combat 
power  would  be  derived  from  the  experience  of  the  Israelis 
in  1973.  The  decisive  factor  which  enabled  the  Israelis  to 
achieve  victory  in  that  war  was  superior  maneuver. 

Critics  of  the  active  defense  doctrine  rapidly  appeared 
both  inside  and  outside  of  the  Army  and  included  members  of 
Congress  such  as  Sen.  Gary  W.  Hart  (D-Colorado) .  Critics  of 
the  doctrine  presented  in  the  1976  version  of  the  FM  100-5 
noted  that  the  manual  presented  an  attrition-based  approach 
to  warfighting.  The  Army  recognized  the  need  to  revise  the 
doctrinal  concepts  in  FM  100-5  and  published  a  new  version 
of  the  manual  in  1982.  In  a  Military  Review  article 
introducing  the  1982  version  of  FM  100-5,  Wass  de  Czege  and 
Holder  (1982)  state  that  the  reason  for  the  publication  of 
the  new  FM  was  simply  that: 

Army  commanders  became  convinced  as  a  result  of 
their  field  training  and  war  games  that  they  would 
be  unable  to  defeat  the  Soviets  using  the  doctrine 
of  1976.  These  commanders  believed  that  they 
could  beat  the  leading  Soviet  echelons  using  the 
"active  defense"  but  that  the  initial  battles 
would  render  our  units  ineffective  while  leaving 
Soviet  follow-on  forces  intact  with  complete 
freedom  of  action.  (p.  53) 

Wass  de  Czege  and  Holder  (1982)  also  note,  however,  that 
the  concepts  presented  in  the  1982  version  of  FM  100-5 
represented  an  evolutionary,  not  revolutionary,  change  in 
doctrine.  That  is,  the  1982  version  of  FM  100-5 
incorporated  the  strong  points  of  the  1976  version, 
reemphasized  maneuver  and  the  principles  of  war  which  had 
been  contained  in  earlier  versions  of  FM  100-5,  and 
presented  new  material  focused  on  the  operational  level  of 
warfighting  and  the  importance  of  integrating  air  and  ground 
forces.  The  doctrine  presented  in  the  1982  version  of  FM 
100-5  has  been  labeled  AirLand  Battle  doctrine. 
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Essentially,  the  1982  version  of  FM  100-5  presented  a 
more  balanced  view  of  the  modern  battlefield  with  a  major 
emphasis  on  the  role  of  maneuver  in  battle.  The  doctrine 
distinguished  between  the  operational  and  tactical  levels  of 
warfighting  and  stressed  the  importance  of  the  operational 
level  to  success  on  the  modern  battlefield.  The  manual  also 
placed  considerable  emphasis  on  the  intangible  aspects  of 
combat  power  such  as  leadership,  cohesion,  and  training. 

The  central  theme  in  the  1982  version  of  FM  100-5  is  the 
AirLand  Battle  operational  concept.  The  manual  stresses  the 
importance  of  initiative  and  bold  actions  to  be  taken  by 
commanders  at  all  levels.  Four  tenets  are  cited  as 
underlying  AirLand  Battle  doctrine:  initiative,  depth, 
agility,  and  synchronization.  The  manual  suggests  that  the 
way  to  win  on  the  modern  battlefield  is  to  seize  and 
maintain  the  initiative  and  shape  the  battle  in  a  proactive 
manner.  The  key  to  seizing  and  maintaining  initiative 
according  to  FM  100-5  is  to  attack  and  fight  the  enemy  in 
depth.  Such  fighting  in  depth  requires  the  synchronization 
of  all  elements  of  combat  power  plus  the  development  of 
mental  and  operational  agility  which  allows  one  to  make 
adjustments  in  plans  and  operations  in  response  to  the 
rapidly  changing  battlefield. 

The  1982  version  of  FM  100-5  touched  upon  the  concept  of 
the  operational  level  of  war  in  only  a  superficial  and 
global  manner.  The  manual  did  note  that  corps-level 
organizations  are  heavily  involved  in  the  operational  level 
of  war fighting.  The  operational  level  was  defined  as  the 
interface  between  strategic  and  tactical  levels  and  involves 
the  planning  and  execution  of  campaigns  and  major  battles 
with  an  emphasis  on  capitalizing  on  the  outcomes  of  tactical 
operations.  The  1982  version  of  FM  100-5  also  noted  that 
corps-level  organizations  were  the  primary  level  for 
coordination  of  information  and  assets  required  to  fight  the 
"deep  battle"  discussed  in  AirLand  Battle  doctrine. 

The  1982  version  of  FM  100-5  and  AirLand  Battle  Doctrine 
has  generated  continued  debate  on  doctrinal  issues  in 
professional  military  publications  such  as  Military  Review. 
A  review  of  the  articles  discussing  these  issues  indicates 
that  the  emphasis  on  maneuver  as  an  element  of  combat  power 
has  been  greeted  with  both  enthusiasm  (Doefel,  1982;  Wass  de 
Czege  and  Holder,  1982)  and  concern  (Maginnis,  1986;  Bates 
and  Quinn,  1986;  Hall,  1986).  The  concern  has  been  voiced 
in  the  form  of  a  call  for  balance  between  emphasis  on 
maneuver  and  firepower  and  the  need  to  recognize  the  demands 
placed  on  leaders  and  units  by  the  new  AirLand  Battle 
doctrine. 
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The  debate  in  the  military  journals  also  indicates  that 
three  aspects  of  the  AirLand  Battle  doctrine  have  stimulated 
a  great  deal  of  interest  and  need  for  clarification.  These 
areas  include  the  concepts  of  operational  level  of 
warfighting,  fighting  the  deep  battle,  and  definition  of  the 
concept  of  synchronization.  In  response  to  the  continued 
doctrinal  debate,  the  Army  has  revised  FM  100-5  again  and 
released  a  new  1986  version  of  the  doctrinal  manual.  This 
latest  version  represents  a  fine-tuning  of  the  1982  version 
which  accomplishes  the  following  goals  (Richardson,  1986) : 

■  The  manual  more  clearly  distinguishes  between  the 

tactical,  operational,  and  strategic  levels  of  war  and 
expands  the  discussion  of  the  operational  level  of  war; 

■  More  fully  explains  the  deep  battle  concept  and  the 

role  of  synchronization  in  combat  operations; 

■  Provides  a  more  balanced  view  of  offensive  and 

defensive  operations; 

■  More  fully  discusses  the  relationship  between  Army  and 
Air  Force  operations  at  the  theater  level; 

■  Clarifies  misperceptions  concerning  the  relationship  of 
AirLand  Battle  doctrine  and  NATO  doctrine. 

The  Role  and  Objectives  of  Army  Corps 

The  1986  version  of  FM  100-5  provides  somewhat  more 
detail  on  the  role  of  the  corps  in  the  AirLand  Battle.  As 
noted  earlier,  the  corps  functions  at  the  interface  of  the 
tactical  and  operational  levels  of  war.  According  to  FM 
100-5  (p.  185): 

The  corps  plans  and  conducts  major  operations  and 
battles.  They  synchronize  tactical  activities 
including  the  maneuver  of  their  divisions,  the 
fire  of  their  artillery  units  and  supporting 
aerial  forces  and  the  actions  of  their  combat 
support  and  CSS  units.  ...  When  employed  alone, 
they  may  exercise  operational  as  well  as  tactical 
responsibilities . 

FM  100-5  also  suggests  that  the  corps  is  the  lowest  level 
at  which  intelligence  and  air  assets  required  to  plan  and 
conduct  major  deep  operations  are  available.  The  corps  and 
division  levels  are  also  the  primary  points  at  which  the 
close,  deep,  and  rear  battles  are  synchronized. 
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More  detailed  information  on  the  role  of  the  corps  in  the  , 
AirLand  Battle  can  be  found  in  FC  100-15  (1984),  a 
preliminary  draft  of  FM  100-15,  Corps  Operations.  According 
to  this  publication,  the  critical  functions  played  by  corps 
in  the  AirLand  Battle  include: 

■  Providing  the  link  between  tactical  operations  and 
strategic  objectives 

■  Fighting  the  enemy  throughout  the  corps'  area  of 
operations  with  maneuver  forces  or  firepower 

■  Maintaining  surveillance  within  and  acquiring 

information  beyond  the  corps'  area  of  operations  to 
provide  an  accurate  picture  of  those  enemy  forces  that 
can  affect  the  current  and  future  battles 

■  Supporting  the  battle  with  combat  support  and  combat 
service  support  forces 

■  Monitoring  closely  air  support  availability  and 
distributing  close  air  support  (CAS)  sorties  for  the 
close-in  battle,  requesting  missions  for  tactical  air 
reconnaissance  (TAR) ,  and  selecting  targets  and 
providing  target  information  for  battlefield  air 
interdiction  (BAI)  missions  within  the  corps  area  of 
operations . 

The  FC  100-15  notes  that  the  corps  commander  "develops 
and  orchestrates  the  operational  plan  to  fight  the  deep 
battle,  the  close-in  battle,  and  the  rear  area  battle 
simultaneously.  He  influences  the  battle  by  dividing  up  the 
battlefield,  through  the  allocations  of  assets,  and  by  the 
synchronization  of  the  AirLand  Battle  within  his  area  of 
responsibility."  (p.  3-8).  The  field  circular  stresses  that 
the  synchronization  of  the  battle  depends  on  subordinates' 
understanding  of  the  commander's  concept  and  notes  that 
synchronization  "is  a  battle  coordination  function 
accomplished  by  the  staff"  (p.  3-15) . 

FC  100-15  suggests  that  the  corps-level  is  the  focal 
point  for  fighting  the  AirLand  Battle  because  AirLand  Battle 
doctrine  is  primarily  an  operational-level  doctrine.  Two 
additional  reasons  are  cited  for  corps  being  the  optimal 
level  for  fighting  the  AirLand  Battle.  First,  the  corps  has 
the  ability  to  continually  gather  and  process  intelligence 
through  its  interfaces  with  national.  Air  Force,  Navy,  and 
organic  surveillance  systems.  Second,  the  corps  appears  to 
be  the  best  level  for  synchronization  of  fires  from  ground, 
air,  and  naval  forces. 
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The  bottom  line  which  can  be  drawn  from  the  review  of 
evolving  Army  doctrine  for  corps  operations  is  that  the 
corps  is  the  primary  level  for  implementation  of  the 
operational  concepts  found  in  the  AirLand  Battle  doctrine 
presented  in  FM  100-5.  Perhaps  the  most  critical  function 
detailed  for  the  corps  is  the  synchronization  of  close-in, 
deep,  and  rear  area  battles  as  well  as  the  synchronization 
of  ground,  air,  and  naval  forces  and  intelligence. 

Interview  Results 

The  findings  from  the  interviews  with  individuals  working 
in  the  area  of  corps-level  doctrine  at  TRADOC  and  CAC 
indicated  that  doctrine  at  this  level  is  in  a  state  of 
transition.  The  interviews  also  indicated  a  recognition 
that  the  issues  surrounding  corps-level  operations  are 
extremely  complex.  Unfortunately,  the  doctrinal  SMEs 
interviewed  had  fairly  limited  personal  experience  at  the 
corps  level.  For  this  reason,  the  decision  was  made  to 
interview  additional  SMEs  with  experience  at  the  corps  level 
and  above. 

The  results  of  the  interviews  with  two  retired  General 
Officers,  both  of  whom  are  noted  for  their  innovative  ideas 
in  the  areas  of  leadership  and  training  at  the  large-unit 
level,  reinforced  much  of  what  was  found  in  the  doctrine 
literature  review.  Both  individuals  noted  that  the  corps  is 
an  extremely  complex  organization  which  presents  major 
challenges  for  its  commander.  The  SMEs  noted  that  the  corps 
commander  needed  a  different  perspective  or  frame  of 
reference  than  that  required  to  command  a  division. 

The  SMEs  noted  that  a  critical  task  which  must  be 
performed  by  the  corps  commander  is  to  clearly  communicate 
his  intent  and  concept  of  an  operation.  In  the  view  of  both 
SMEs,  this  is  an  essential  ingredient  for  the  execution  of 
synchronized  operations.  One  noted  that  the  corps  commander 
had  to  ingrain  his  intent  in  his  staff  and  essentially  shape 
and  mold  his  staff  to  fit  his  style  of  leadership  and 
concept  of  operations.  Both  SMEs  also  suggested  that  one  of 
the  most  critical  performance  factors  to  be  considered  in 
examining  corps  performance  is  the  corps'  ability  to  process 
information  in  a  timely  and  accurate  manner. 

Summary 

After  completing  the  interviews  with  the  corps-level  SMEs 
and  finishing  the  review  of  Army  doctrine  and  related 
publications,  the  research  team  had  successfully  identified 
the  general  objectives  of  corps-level  organizations  within 
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the  context  of  AirLand  Battle  doctrine.  The  review  of 

doctrine  indicated  that  some  of  the  key  objectives  to  be 
performed  by  corps-level  organizations  include: 

■  Synchronization  of  the  close-in,  deep,  and  rear  area 
battles 

■  Fighting  the  enemy  throughout  the  corps '  area  of 

operations 

■  Synchronization  of  air,  ground,  and  naval  forces 

■  Maintaining  surveillance  within  and  acquiring 

information  beyond  the  corps’  area  of  operations  to 
provide  an  accurate  picture  of  enemy  forces 

■  Support  the  battle  with  combat  support  and  combat 

service  support  forces 

While  the  objectives  described  above  are  not  assumed  to 
be  exhaustive  and  are  primarily  focused  on  the  corps' 
wartime  mission,  they  were  deemed  as  adequate  to  provide  the 
basis  for  further  development  of  the  corps  performance 
measurement  system  for  implementation  within  the  context  of 
the  joint  exercises  conducted  by  REDCOM.  The  next  step  in 
the  development  of  the  corps  performance  measurement  system 
was  to  identify  potential  outcome  measures  related  to  the 
objectives  identified  in  the  literature  review  and 
interviews. 
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SECTION  4 

POTENTIAL  CRITERION  MEASURES  AND  A  CORPS  PERFORMANCE  MODEL 


General 

The  second  and  third  steps  in  the  development  of  an 
organizational  performance  measurement  system  are  the 
identification  of  outcome  measures  related  to  the  objectives 
and  purpose  of  the  organizational  element  of  interest  and 
the  development  of  a  performance  model  which  identifies 
individual ,  organizational ,  and  environmental  factors  which 
impact  on  these  performance  outcomes.  In  the  case  of 
complex  organizational  elements,  it  may  be  necessary  to 
perform  these  two  steps  in  tandem.  The  research  team  in  the 
present  effort  found  this  to  be  the  case  in  the  attempt  to 
identify  performance  outcomes  for  Army  corps. 

Potential  Criterion  Measures 

Examination  of  the  objectives  identified  at  the  end  of 
Section  3  suggests  that  outcome  measures  related  to  the 
corps'  primary  objectives  may  be  classified  into  three 
related  categories:  The  first  category  is  combat  outcome 
measures  which  are  related  to  the  "bottom  line"  of  mission 
accomplishment.  These  measures  would  include  those  related 
to  casualties  inflicted  on  the  enemy,  casualties  suffered, 
territory  captured,  etc.  The  second  category  of  outcome 
measures  would  be  measures  related  to  logistics  and 
efficient  distribution,  management,  and  use  of  equipment, 
supplies,  etc.  This  second  category  essentially  represents 
an  indicator  of  efficiency  of  corps  operations  while  the 
first  is  more  directly  related  to  overall  effectiveness  of 
the  operation.  The  final  category  of  measures  or  products 
are  those  related  to  information  processing  and  intelligence 
analysis.  A  number  of  information-related  products  are 
required  in  planning  and  executing  a  synchronized  operation. 
Considering  the  major  emphasis  on  corps-level 
synchronization  of  various  aspects  of  combat  operations, 
evaluation  of  such  information  products  and  processes 
represents  a  critical  dimension  of  corps  performance. 

Details  of  the  nature  of  the  measures  and  types  of  data 
to  be  collected  for  the  outcome  measures  noted  above  will  be 
presented  in  a  later  section  of  the  report.  At  this  point, 
it  is  simply  important  to  note  that  assessment  of  the  final 
outcome  of  corps-level  performance  will  require  multiple 
criterion  measures.  Furthermore  potential  criterion 
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measures  fall  into  the  categories  of  battle  outcome 
measures,  logistics  and  supply  efficiency  measures,  and 
information  products  and  processes. 

A  major  problem  in  dealing  with  criterion  measures  for 
an  organization  such  as  an  Army  Corps  is  the  fact  that 
bottom-line  outcome  measures  such  as  combat  outcomes 
represent  the  effects  of  actions  taken  at  various  echelons 
j.n  the  organization.  As  illustrated  in  Figure  4-1,  bottom- 
line  outcome/combat  outcome  measures  reflect  the  nested 
effects  of  corps,  division,  brigade,  battalion,  company,  and 
platoon  actions.  The  exact  nature  of  the  effects  produced 
at  each  level  and  the  relationships  between  effects  across 
levels  is  largely  unknown.  One  can  probably  suggest  that 
the  effects  of  planning  and  distribution  of  resources  have 
their  impact  on  combat  outcomes  in  primarily  a  top-down 
manner.  On  the  other  hand,  effects  which  occur  as  a  result 
of  errors  in  execution  are  likely  to  have  their  effects  in  a 
bottom-up  manner.  This  is  not  to  say  that  errors  in 
execution  of  a  mission  at  corps  level  will  not  have  a  major 
impact.  Rather,  it  suggests  that  the  inability  of 
individual  soldiers  or  small  units  to  perform  their  missions 
will  spell  disaster  for  even  the  best-planned  operations. 
Likewise,  outstanding  performance  at  the  individual  and 
small-unit  level  will  be  wasted  if  such  efforts  do  not  occur 
as  part  of  a  synchronized  plan  which  has  been  well 
coordinated  and  resourced  from  upper  command  levels. 

If  the  organizational  element  of  interest  is  the  corps 
staff,  intermediate  outcome  measures  which  are  related  to 
battle  outcomes  should  be  identified.  The  intermediate 
performance  indicators  should  be  directly  related  to  the 
corps  staff's  actions  which  impact  on  the  bottom-line 
battlefield  outcomes  through  the  actions  of  echelons  below 
the  corps  or  through  elements  of  other  organizations  not 
organic  to  the  corps. 

A  Corps  Performance  Model 

The  development  of  a  multi-echelon  model  of  combat 
performance  for  all  levels  at  or  below  corps  is  a  task 
beyond  the  scope  of  the  current  effort.  For  this  reason, 
the  use  of  bottom-line  combat  outcome  measures  as  the 
primary  measure  of  corps  performance  is  unlikely  to  provide 
the  type  of  information  required  for  diagnostic  feedback  for 
training  purposes.  This  is  not  to  say  that  such  outcome 
measures  are  not  important.  Instead,  it  suggests  that 
considerable  research  is  required  to  provide  a  more  complete 
model  of  combat  performance  to  sort  out  the  contribution  of 
actions  at  various  echelons  to  final  combat  outcomes. 
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FIGURE  4-1 .  Nested  Unit  Performance  Effects 
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ERRORS  IN  EXECUTION 


A  more  realistic  approach,  and  the  one  adopted  in  the 
current  effort,  is  to  focus  attention  on  those  products  or 
outcomes  which  directly  reflect  the  actions  of  individuals 
at  the  corps  level.  The  review  of  doctrine  and  the 
discussion  above  suggests  that  the  appropriate  measures  of 
interest  in  assessing  corps-level  performance  would  be 
measures  related  to  planning,  information  processing, 
coordination,  and  resource  allocation.  Furthermore,  a  model 
of  corps-level  performance  which  focused  on  direct  outcomes 
of  corps-level  actions  would  be  primarily  an  information 
processing  model. 

Figure  4-2  provides  an  illustration  of  a  greatly 
simplified  model  of  information  processing  dynamics  at  the 
corps  level.  The  model  suggests  that  factors  affecting  the 
performance  of  the  corps  include  accuracy  of  information 
coming  into  the  corps  from  echelons  above  the  corps,  other 
services,  organic  corps  intelligence  assets,  and  spot 
reporting  by  subordinate  tactical  elements  in  the  corps. 
Furthermore,  individual  and  organizational  communication, 
information  processing,  and  decision  making  variables  will 
impact  on  performance  by  affecting  the  quality,  timeliness, 
etc.  of  information  processing  within  the  corps  staff 
itself.  As  shown  in  Figure  4-2  all  of  these  variables 
should  impact  on  the  initial  plan  and  subsequent  ability  of 
the  corps  staff  to  coordinate  execution  of  a  planned 
operation.  These  two  factors  will,  in  turn,  impact  on  the 
degree  of  synchronization  which  is  displayed  in  the  actual 
execution  of  the  operation. 

The  successful  execution  of  planned  events  in  an 
operation  will  reflect  the  effects  of  all  echelons  down  to 
the  platoon  and  squad  level  in  actual  combat.  In  CPXs  and 
other  combat  simulation  exercises,  the  outcomes  are  the 
products  of  the  effects  down  to  the  lowest  level  players  and 
the  algorithms  used  in  combat  models  and  simulations. 
However,  the  degree  to  which  a  plan  provides  for 
synchronized  actions,  the  degree  to  which  the  corps 
commander's  intent  and  concept  of  the  operation  is 
communicated  to  the  division  level,  the  degree  to  which 
timely  information  is  provided  to  division  level,  and  the 
extent  to  which  adequate  resources  are  distributed  to 
division  level  are  direct  measures  of  the  performance  of  a 
corps  commander  and  his  staff. 

Figure  4-3  presents  a  modified  version  of  the  corps 
performance  model  which  focuses  on  synchronization  of 
operations  as  the  primary  outcome  measure  of  corps 
performance.  In  this  model,  the  initial  factor  impacting  on 
corps  performance  is  the  information  provided  to  the  corps 
commander  and  the  corps  G-3,  by  the  G-2  and  EAC.  The  G-3 
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FIGURE  4-2.  Initial  Corps  Performance  Model 
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PLAN 

FIGURE  4-3.  Corps  Synchronization  Performance  Model 


and  the  corps  commander  are  the  primary  planners  for  the 
corps  operation.  These  two  individuals  also  share  the 
greatest  burden  in  communication  of  the  the  concept  of  the 
operation  and  the  commander's  intent  to  the  division 
commanders  and  the  corps  staff.  Once  the  operations  begin, 
the  corps  staff  essentially  functions  as  an  extension  of  the 
commander  in  communicating  and  executing  his  intent  and 
concept  of  the  operation.  Once  the  execution  phase  is 
entered,  the  ability  of  the  corps  staff  to  gather  and 
process  information  is  the  primary  corps-level  effect  on 
synchronization  of  the  operation.  Subordinate  units  impact 
on  synchronization  through  their  success  and  failure  in 
executing  their  respective  missions  outlined  in  the  plan  and 
subsequent  orders  provided  by  the  corps  commander  and  his 
principal  staff. 

In  the  model  described  above,  synchronization  is 
treated  as  a  surrogate  measure  of  combat  performance.  While 
the  ultimate  criteria  of  interest  are  combat  outcome 
measures,  as  noted  earlier,  such  criterion  measures  are 
"contaminated"  by  a  variety  of  nested  unit  effects  which  are 
not  clearly  understood.  As  a  result,  the  proposed 
measurement  system  will  use  measures  of  synchronization  and 
direct  measures  of  corps  staff  actions  as  performance 
indicators.  As  the  measurement  system  is  developed  and  data 
are  collected,  it  will  be  possible  to  examine  the 
relationships  between  these  "proximal"  criteria  and  the 
"distal"  criterion  of  combat  outcome. 

The  strategy  of  using  measures  of  synchronization  as  a 
proximal  outcome  measure  is  not  considered  to  be  the  optimal 
solution  to  the  corps  performance  measurement  problem. 
However,  given  the  results  of  the  review  of  AirLand  Battle 
doctrine  presented  earlier  and  the  findings  from  a  review  of 
existing  literature  on  organizational  performance 
measurement  systems,  it  represents  the  most  viable  approach 
given  the  current  state-of-the-art  in  performance  measure¬ 
ment  technology. 

The  review  of  the  performance  measurement  literature 
indicates  that  an  information  processing  model  of  corps 
performance  is  consistent  with  the  models  used  in  previous 
attempts  to  measure  performance  01  commanders  and  their 
staffs.  The  review  of  existing  measurement  systems 
developed  to  assess  performance  of  Army  staffs  provides 
evidence  supporting  the  information  processing  approach 
adopted  in  the  current  effort.  The  review  also  identifies  a 
number  of  problems  which  must  be  avoided  in  development  of 
the  current  performance  measurement  system. 
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The  performance  measurement  systems  literature  review 
focused  on  the  FORGE  research  program  (Olmstead,  1978),  the 
Headquarters  Effectiveness  Assessment  Tool  (HEAT) 
methodology  (HEAT  Executive  Summary,  1984) ,  the  NTC 
performance  measurement  research  program  (The  Army  Research 
Institute  Plan  for  an  NTC-Based  Research  Program,  1986) ,  and 
combat  modeling.  A  more  detailed  discussion  of  the 
literature  review  may  be  found  at  Appendix  B.  The  review  of 
the  performance  measurement  literature  and  examination  of 
each  of  the  four  measurement  approaches  listed  above 
suggests  several  conclusions.  First,  there  is  evidence  that 
information  processing  models  of  command  and  staff 
performance  may  be  related  to  effective  performance  of 
combat  units.  Second,  while  information  process  measures 
may  be  important,  process-oriented  measurement  systems  tend 
to  produce  highly  subjective  data  which  are  not  outcome- 
oriented.  Third,  a  performance  measurement  system 
resembling  the  outcome-oriented  system  under  development  for 
the  NTC  has  more  potential  for  acceptability  and  utility 
than  a  performance  measurement  system  which  is  primarily 
process  oriented. 

Another  lesson  learned  by  the  present  authors  from  work 
conducted  at  the  NTC  is  that  simply  having  large  amounts  of 
objective  data  does  not  ensure  that  commanders  and  their 
staffs  will  receive  effective  feedback.  In  fact,  work  at 
the  NTC  indicates  that  an  effective  performance  measurement 
system  must  be  developed  based  on  some  model  of  unit 
performance  rather  than  developed  inductively  by  aggregating 
large  amounts  of  unrelated  data  which  have  been  collected 
simply  because  an  instrumentation  system  made  it  feasible  to 
collect. 

The  review  of  the  performance  measurement  literature 
also  suggests  that  some  caution  must  be  exercised  in  using 
combat  outcome  measures  generated  by  combat  models.  This 
finding  reinforces  the  warning  voiced  earlier  concerning  the 
use  of  bottom-line  combat  outcome  measures  as  the  primary 
criterion  measure  for  assessing  corps-level  performance. 

The  Corps  Model  of  Performance  Revisited 

The  review  of  the  measurement  approaches  described  in 
Appendix  B  suggests  that  the  model  of  corps  performance 
presented  in  Figure  4-3  represents  a  reasonable  basis  for 
identification  of  criterion  measures  and  performance  factors 
for  the  corps  level.  Examination  of  the  model  in  Figure  4-3 


4-8 


suggests  a  number  of  potential  factors  which  may  impact  on  a 
corps  performance  as  measured  by  its  ability  to  conduct  a 
synchronized  combat  operation.  The  factors  include: 

■  The  accuracy  and  completeness  of  the  information  used 
to  develop  the  corps'  operations  plan 

■  The  soundness ,  adequacy,  etc.  of  the  operations  plan 
developed  by  the  commander  and  the  G-3 

■  The  degree  to  which  the  plan  and  intent  of  the 

commander  are  clearly  communicated  and  understood  by 
subordinate  commanders  and  the  corps  staff 

■  The  degree  to  which  the  corps  staff  successfully 

accomplishes  the  analysis,  synthesis,  and  communication 
of  information  required  to  develop  and  implement  the 
plan 

■  The  extent  to  which  subordinate  echelons  (particularly 
the  maneuver  elements)  accomplish  their  assigned 
missions 

■  The  degree  to  which  lateral  and  vertical  communications 
are  maintained  during  execution  of  the  operation 

■  The  ability  of  the  commanders  and  staffs,  particularly 

at  upper  levels  (division  and  corps)  to  adjust  the 

original  plan  in  response  to  changes  in  the  battle 

scenario 

■  Logistical  support  to  maneuver  elements 

While  the  list  above  is  not  exhaustive,  it  does  provide  a 
number  of  key  performance  dimensions  which  should  impact 
significantly  on  the  ability  of  the  corps  to  accomplish  its 
assigned  mission.  These  performance  dimensions  can  be 
directly  translated  into  measurement  requirements  which 
identify  the  type  of  performance  data  which  must  be 

collected  to  provide  diagnostic  feedback  to  the  corps 

commander  and  his  staff  for  training  purposes.  Table  4-1 
lists  the  criterion  data  and  performance  dimension  data 
categories  which  the  review  of  doctrine  and  corps 
performance  model  suggest  should  be  included  in  the  corps 
performance  measurement  system. 
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TABLE  «-1.  OATA  CATEGORIES 


CRITERION  OATA 

BATTLE  OUTCOMES: 

FRIENDLY  CASUALTIES 

FRIENDLY  BATTLE  DAMAGE  ASSESSMENT 
(BOA) 

OPFOR  CASUALTIES 

OPFOR  BOA 

TERRAIN  GAINED /LOST 

TIME  FACTORS 

LOGISTICS: 

RESOURCES  EXPENDED 

RESOURCES  DISTRIBUTED 

ETC. 

PERFORnANCE 

SYNCHRONIZATION  MEASURES: 

DATA 

PLANNED  EVENTS 

ACTUAL  EVENTS 

PLANNED  VS.  ACTUAL  TIME 

PLANNED  VS.  ACTUAL  LOCATION 

INFORMATION 

COMMUNICATIONS 

PROCESSING 

TYPE  INFORMATION  PROCESSED 

TIME  TO  PROCESS  INFORMATION 

SITUATION  MAP  VS.  GROUND  TRUTH 

TIME  TO  DISSEMINATE  INFORMATION 
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The  next  section  of  the  report  presents  a  definition 
and  discussion  of  the  concept  of  synchronization. 
Synchronization  is  one  of  the  underlying  tenets  of 
AirLand  Battle  doctrine  and  is  the  primary  construct 
underlying  the  performance  measurement  concept 
developed  in  the  current  research  effort.  Following 
the  discussion  of  the  synchronization  concept  is  a 
detailed  description  of  the  Inchon  Landing  Operation 
conducted  during  the  Korean  War.  The  Inchon  Landing 
example  is  used  to  illustrate  a  means  by  which  synchro¬ 
nization  of  operations  may  be  illustrated  graphically. 
The  graphic  representation  of  synchronization  is  one 
facet  of  the  measurement  development  concept  which  will 
be  presented  in  detail  following  the  Inchon  landing 
example . 
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SECTION  5 

A  MEASUREMENT  CONCEPT  FOR  SYNCHONIZATION  IN  CORPS  OPERATIONS 


Introduction 

The  complexity  of  a  corps-level  operation  requires  a 
performance  measurement  system  to  impose  a  certain  degree  of 
structure  on  the  activities  taking  place  during  the 
operation.  The  concepts  underlying  the  development  of  the 
measurement  system  are  the  primary  source  of  this  structure. 
This  section  of  the  report  provides  a  working  definition  of 
the  concept  of  synchronization  in  combat  operations.  The 
synchronization  concept  is  the  primary  performance  construct 
underlying  the  measurement  concept  described  in  the 
remainder  of  the  report. 

Definition  of  Synchronization 

FM  100-5,  Operations  (1986) ,  defines  synchronization  as 
'•the  arrangement  of  battlefield  activities  in  time,  space 
and  purpose  to  produce  maximum  relative  combat  power  at  the 
decisive  point."  (p.  17)  It  should  be  noted,  however,  that 
synchronization  is  a  much  more  complex  concept  than  that  of 
massing  fires  on  an  objective.  As  noted  in  FM  100-5, 
synchronized  activities  may  be  "separate  in  time  and  space; 
however,  these  activities  are  synchronized  if  their  combined 
consequences  are  felt  at  the  decisive  time  and  place."  (p 
17)  For  example,  if  actions  taken  to  disrupt  enemy  supply 
lines  through  deep  attacks  produce  shortages  of  critical 
supplies  at  the  time  of  a  decisive  engagement  in  the  close 
battle,  then  the  activities  of  the  rear  and  close  actions 
have  had  their  desired  synchronized  effects. 

The  combined  consequences  of  synchronized  activities 
may  produce  tactical,  logistical,  and/or  psychological 
effects.  In  fact,  optimally  synchronized  actions  such  as 
the  Inchon  Landing  engineered  by  General  Douglas  MacArthur 
during  the  Korean  War  are  probably  successful  because  the 
effects  they  produce  on  enemy  forces  are  both  psychological 
and  physical  in  nature.  For  example,  actions  designed  to 
produce  shortages  in  enemy  supplies  are  likely  to  be  even 
more  successful  if  they  create  the  perception  that  forces 
near  the  FLOT  are  cut-off  or  surrounded.  If  the  commander 
can  time  a  major  thrust  at  the  FLOT  with  the  arrival  of 
information  (or  misinformation)  in  enemy  intelligence 
channels  that  the  units  near  the  FLOT  have  been  cut-off,  the 
combined  consequences  of  these  activities  will  represent  a 
synchronized  operation. 

While  not  stated  as  such,  an  underlying  theme  found  in 
the  many  discussions  of  synchronization  is  that  synchronized 
activities  produce  synergistic  effects  which  are  greater 
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than  the  sum  of  the  identical  activities  carried  out  in  an 
unsynchronized  manner.  Such  synergistic  effects  lie  at  the 
core  of  AirLand  Battle  doctrine  as  presented  in  FM  100-5. 
As  noted  in  FM  100-5 ,  to  execute  a  synchronized  operation  a 
commander  and  his  staff  must  construct  and  clearly 
communicate  a  detailed  plan  of  the  operation.  The 
foundation  for  synchronized  operations  must  be  laid  in  the 
planning  and  preparation  phases  of  the  mission  because 
positive  command  and  control  becomes  extremely  difficult 
once  an  operation  enters  the  execution  phase. 

General  William  DePuy,  USA  retired,  (1984)  has 
suggested  that  synchronization  is  a  complementary  function 
which  provides  balance  in  a  bold  maneuver-based  doctrine. 
DePuy  uses  the  term  synchronization  to  refer  to  horizontal 
coordination  across  functional  control  areas  such  as 
maneuver,  fire  control,  and  tactical  air  support.  DePuy 
contrasts  synchronization  to  the  vertical  functional  control 
process  across  echelons.  DePuy  also  contrasts 
synchronization  and  maneuver  as  being  two  methods  for 
concentrating  combat  power  —  maneuver  is  the  method  for 
concentration  of  forces  in  space  while  synchronization  is 
the  method  for  concentrating  actions  in  time.  In  DePuy' s 
conceptualization,  synchronization  is  enhanced  through 
detailed  planning  and  clear  communication  of  the  commander's 
intent . 

More  recent  use  of  the  term  synchronization  encompasses 
the  concept  of  concentration  in  both  the  time  and  space 
dimensions  with  coordinated  maneuver  considered  as  one 
aspect  of  synchronization.  DePuy 's  model  is  important, 
however,  in  that  it  suggests  that  synchronization  must  be 
viewed  as  having  both  horizontal  and  vertical  dimensions. 
The  vertical  dimension  of  "functional  control"  between 
echelons  is  most  closely  related  to  current  definitions  of 
command  and  control.  The  horizontal  "synchronization" 
across  functional  areas  represents  staff  integration  at  each 
echelon.  DePuy 's  model  suggests  that  each  echelon 
represents  an  information  synthesis  node  for  horizontal 
coordination  across  functional  areas  while  information  flow 
between  echelons  must  follow  the  chain  of  command  within 
each  functional  area.  If  Figure  5-1  is  viewed  as  an 
information  flow  matrix,  DePuy  suggests  that  information  can 
flow  only  through  horizontal  and  vertical  pathways.  An 
alternative  proposition  may  be  that  thorough  understanding 
of  the  commander's  intent  and  efficient  staff  functioning 
should  allow  information  flow  along  diagonal  lines,  i.e., 
across  both  functional  areas  and  echelons  simultaneously. 
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FIGURE  5-1.  Horizontal,  Vertical  And  Time  Dimension 
Associated  With  Planning  And  Executing  A 


Such  information  flow  patterns  may  be  required  to  execute 
AirLand  Battle  doctrine  and  pass  information  upward  in  a 
timely  manner.  Perhaps,  the  existence  of  effective  diagonal 
flows  ^f  information  is  one  indicator  of  synchronization  in 
staff  processes. 

While  DePuy  suggests  detailed  planning  as  the  primary 
means  of  producing  synchronized  operations,  he  also  notes 
that  effective  communication  of  the  commander's  intent  is  a 
critical  factor.  In  an  ideal  sense,  a  synchronized 
operation  is  one  in  which  the  commander's  intent  is 
successfully  operationalized  on  the  battlefield.  Whether  or 
not  such  a  synchronized  operation  leads  to  successful 
outcomes  depends  in  part  on  the  quality  of  the  commander's 
concept  of  operation.  A  well  synchronized  operation  may 
still  result  in  failure  if  the  commander's  concept  is  based 
on  inaccurate  information  or  if  unpredictable  events  such  as 
enemy  use  of  nuclear  weapons  occur.  Furthermore,  the 
ability  of  the  units  involved  in  the  operation  to  adjust  to 
unexpected  events  will  also  impact  on  success. 
Synchronization  in  the  face  of  unexpected  events  on  the 
battlefield  requires  complete  understanding  of  the  senior 
commander's  intent  as  well  as  some  minimum  level  of 
communications  across  both  echelons  and  functional  areas. 

Moving  from  the  Abstract  to  the  Concrete 

The  definition  of  synchronization  described  above  is  a 
working  definition  based  on  concepts  discussed  in  current 
military  literature.  To  further  clarify  exactly  what  is 
meant  by  the  term  synchronization  and  to  examine  a  potential 
means  for  measuring  synchronization,  the  research  team 
examined  historical  examples  of  synchronized  operations. 
After  a  measurement  concept  for  synchronization  was 
developed,  the  concept  was  applied  to  the  analysis  of  the 
Inchon  Landing  Operation  conducted  by  General  Douglas 
MacArthur  during  the  Korean  War. 

The  Inchon  Landing  analysis  provides  a  concrete  example 
of  the  synchronization  concept.  This  example  will  be 
presented  in  the  next  section  to  illustrate  how 
synchronization  in  an  actual  military  operation  can  be 
illustrated  graphically.  Following  the  Inchon  Landing 
example,  the  general  performance  measurement  concept  for 
assessing  corps-level  operations  within  a  joint  exercise 
context  will  be  described  in  detail. 


5-4 


SECTION  6 

APPLICATION  OF  A  SYNCHRONIZATION  MEASUREMENT  CONCEPT 
TO  THE  INCHON  LANDING  OPERATION 


Introduction 

One  means  by  which  the  authors  examined  the  viability 
of  the  synchronization  measurement  concept  was  to  apply  the 
concept  to  the  Inchon  Landing,  a  corps-level  operation 
^conducted  during  the  Korean  War.  The  purpose  of  the 
application  was  to  examine  the  results  of  using  the 
battlefield  event  timelines  as  a  means  of  representing 
planned  and  actual  events  in  a  synchronized  major  operation. 
While  the  detailed  operations  plan  from  which  the  planned 
event  timeline  would  normally  be  constructed  was  not 
available,  it  was  possible  to  construct  fairly  detailed 
timelines  of  planned  events  based  on  available  historical 
accounts  of  the  Inchon  Landing. 

Approach 

General  Douglas  MacArthur's  invasion  of  Inchon 
(Operation  Chromite)  was  selected  as  the  vehicle  for 
exploring  the  application  of  the  synchronization  measurement 
concept  for  two  reasons:  1)  the  degree  of  planning  and 
precision  involved,  and  2)  the  operation  demonstrated 
success  in  employing  numerous  combat  elements  at  various 
echelons.  As  discussed  previously,  the  measurement  approach 
applied  to  the  Inchon  example  was  based  largely  on:  1)  The 
FM  100-5  (1986)  definition  of  synchronization  (particularly 
elements  of  time,  space  and  purpose)  and  2)  General  Will  m 
DePuy's  (1984)  view  of  military  organizations  as  verticaxxy 
and  horizontally  integrated  functional  systems.  A  review 
of  Michael  Langway 's,  Inchon:  MacArthur's  Last  Triumph 
(1979)  and  other  sources  (Hoyt,  1984;  Sheldon,  1968; 
Manchester,  1978)  revealed  that  sufficient  detail  regarding 
time,  space  and  purpose  of  many  of  the  Inchon  combat 
activities  were  available  to  conduct  the  analysis. 

Evolution  of  Operation  Chromite 

Before  discussing  the  development  of  the  Inchon  Landing 
example,  it  is  useful  to  place  this  ambitious  assault  in  the 
proper  context  of  the  Korean  War.  The  invasion  of  South 
Korea  occurred  on  25  June  1950,  when  North  Korean  tank 
forces  crossed  the  38th  parallel  (See  map  at  Figure  6-1) . 
At  that  time  there  was  only  a  small  advisory  force  of 
American  troops  in  South  Korea.  The  Korean  Army  itself  was 
ill-prepared.  Armed  with  the  element  of  surprise,  the  North 
Korean  People's  Army  (NKPA)  made  quick  and  strong  advances 
and  it  was  not  until  late  July  that  any  effective  resistance 


6-1 


could  be  mounted.  Essentially,  North  Korean  forces  had 
overrun  the  country,  being  stopped  just  short  of  Pusan  at 
the  southern  end  of  the  peninsula  by  the  Eighth  Army 
recently  re-established  under  authority  of  the  United 
Nations,  on  10  September,  the  NKPA  advance  was  halted; 
however,  NKPA  deployment  of  an  additional  98,000  men  in  13 
infantry  divisions  threatened  to  defeat  the  Eighth  Army. 
Five  days  later  General  Douglas  MacArthur  initiated 
Operation  Chromite  with  a  joint  force  tailored  to  perform  an 
amphibious  assault  on  Inchon.  In  Reminiscences  (1964)  the 
general  described  his  operation  and  his  intent  in  the 
following  cable  sent  to  Washington  on  23  July  1950: 

Operation  planned  mid-September  is  amphibious 
landing  of  a  two  division  corps  in  rear  of  enemy 
lines  for  purpose  of  enveloping  and  destroying 
enemy  forces  in  conjunction  with  attack  from  south 
by  Eighth  Army.  I  am  firmly  convinced  that  early 
and  strong  effort  behind  his  front  will  sever  his 
main  lines  of  communication  and  enable  us  to 
deliver  a  decisive  and  crushing  blow.  The 
alternative  is  a  frontal  attack  which  can  only 
result  in  a  protracted  and  expensive  campaign,  (p. 

346) 

On  29  August  1950,  MacArthur  was  notified  by  wire  from 
the  Joint  Chiefs  of  Staff  to  begin  preparations  for  the 
Inchon  landing.  Planning  had  already  begun  by  ea^ ly-August 
but,  "The  tide  and  the  treacherous  channel,  blockaded  by  the 
island,  gave  MacArthur  six  weeks  preparation  and  training  in 
which  thoroughly  to  familiarize  his  men  with  the  curious 
habits  of  the  influx,  instead  of  the  six  months  which  such 
an  operation  would  normally  have  demanded."  (Langley,  p.  20) 
While  Operation  Chromite  was  being  planned  in  detail  by 
General  Almond  of  X  Corps,  the  command  structure  (shown  in 
Figure  6-2)  was  being  assembled.  MacArthur  had  continued  to 
argue  his  case  that  the  NKPA  could  not  prepare  to  repel  an 
assault  in  so  short  a  period  of  time,  if  in  fact  the  enemy 
even  believed  such  an  operation  would  be  undertaken.  He  was 
later  proven  correct.  When  the  Marines  landed  at  Inchon  on 
15  September,  the  NKPA  was  caught  by  surprise  by  American 
troops  who  demonstrated  a  keen  understanding  of  MacArthur 's 
intent  and  concept  for  the  Inchon  Landing  operation. 

The  Inchon  invasion  consisted  of  three  distinct 
amphibious  landings,  the  latter  two  occurring  simul¬ 
taneously.  Each  landing  occurred  on  a  separate  beach, 
designated  by  one  of  the  colors  green,  red  or  blue.  A  map 
of  the  Inchon  area  at  Figure  6-3  shows  the  location  of  the 
invasion  beaches.  Preceding  the  operation,  information  was 
gathered  by  the  Navy  which  detailed  the  landing  conditions 


6-3 


FIGURE  6-2.  Command  And  Control  -Structure  For 
Operation  Chromite  . 
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the  Marines  would  encounter,  and  provided  some  knowledge  of 
the  nature  of  the  defenses  at  Inchon.  Green  Beach  was  the 
first  to  be  assaulted  at  0633  on  15  September;  however,  the 
actual  landing  was  preceded  by  a  series  of  intense  naval  and 
aerial  bombardments  over  the  course  of  several  days  that 
significantly  softened  NKPA  defenses.  When  the  3rd 
Battalion  of  the  5th  Marine  Division  went  ashore  at  Green 
Beach  they  were  met  with  light  resistance.  By  1215  they  had 
gained  full  control  of  the  beach. 

Shortly  before  Green  Beach  was  officially  captured, 
U.S.  aerial  and  naval  bombardment  of  Red  Beach  was  ordered, 
this  time  with  the  additional  goal  of  sealing  off  the  Inchon 
area.  At  1430  British  warships  began  shelling  Blue  Beach. 
Both  remaining  beaches  were  scheduled  to  be  assaulted  at 
1730.  The  1st  and  2nd  Battalion  of  the  5th  Marine  Division 
arrived  on  Red  Beach  three  minutes  late  and  experienced  some 
difficulty  getting  all  troops  ashore.  Nonetheless,  Red 
Beach  was  secured  at  2359,  though  more  resistance  had  been 
encountered  than  at  Green  Beach. 

Blue  Beach  operations,  which  began  on  time,  were 
intended  to  secure  the  southern  and  only  remaining  approach 
route  to  the  port.  As  with  the  other  landings,  Blue  Beach 
was  preceded  by  heavy  shelling.  Two  beaches  were  assigned 
to  the  2nd  and  3rd  Battalions  of  the  1st  Marine  Division 
respectively.  The  landing  proceeded  somewhat  chaotically: 
some  equipment  and  troops  arrived  at  the  wrong  place;  some 
troops  arrived  late;  and,  in  the  midst  of  this,  a  third 
impromptu  beach  was  established.  Fighting  continued  until 
shortly  after  midnight,  when  Blue  Beach  was  finally  secured. 
In  all,  13,000  Marines  had  been  put  ashore  that  day;  Inchon 
and  its  port  access  was  under  U.S.  control;  and  Marine 
casualties  were  relatively  light  (21  killed,  174  wounded) . 

The  result  of  Operation  Chromite  was  the  swift  capture 
of  Inchon,  and  subsequently,  the  capture  of  Seoul  13  days 
later.  These  events  in  turn  prompted  redeployment  of  NKPA 
resources  and  enabled  the  Eighth  Army  to  break  out  from  the 
Pusan  perimeter.  As  forces  from  MacArthur's  X  Corps  and  the 
Eighth  Army  converged,  the  NKPA  were  caught  in  the  middle 
and  125,000  prisoners  were  taken;  the  initiative  had  been 
captured  by  MacArthur  and  the  U.N.  forces.  What  MacArthur 
and  the  U.N.  forces  did  not  know  was  that  a  few  Chinese 
units  had  begun  advancing  into  North  Korea  shortly  after 
receiving  word  of  the  Inchon  invasion.  By  the  end  of  1950, 
the  number  of  Chinese  and  NKPA  forces  had  grown  to  500,000 
and  ultimately  they  ended  the  advance  of  U.N.  forces. 
MacArthur's  recall  by  President  Truman  was  to  follow  a  few 
months  later.  These  latter  events,  however,  occurred  well 
after  the  effects  of  Inchon  had  been  felt. 
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Methodology 


As  shown  in  Section  5,  a  hypothetical  matrix  was 
constructed  (Figure  6-4)  to  depict  the  vertical,  horizontal 
and  time  dimensions  associated  with  planning  and  executing 
an  operation.  This  is  the  first  step  in  the  development  of 
the  synchronization  example.  Each  cell  within  the  matrix 
represents  a  functional  element  within  the  military 
organization  at  a  given  point  in  time.  The  time,  space,  and 
purpose  associated  with  each  activity  are  initially 
established  by  the  commander  (CMDR)  and/or  his  staff.  They 
may  be  referenced  in  the  commander's  explanation  of  the 
operation  (CMDR's  intent)  or  included  in  the  operation 
orders.  The  degree  of  synchronization  achieved  by  the 
organization,  including  any  adjustments  required  during  the 
course  of  an  operation,  is  then  determined  by  (1)  The  degree 
of  effective  horizontal  coordination  over  time,  and  (2)  The 
ability  of  individual  functional  elements  to  perform 
assigned  activities  successfully,  i.e.,  achieving  the 
objective  at  the  right  place  and  time. 

The  second  step  in  development  of  the  synchronization 
example  was  to  construct  a  timeline  based  upon  the  key 
dimensions  of  interest:  horizontal  coordination  across 
functional  area,  and  time.  Composed  of  details  from 
Operation  Chromite,  Green  Beach  landing,  Figure  6-5  shows 
the  actions  of  each  functional  element.  Time  and  functional 
areas  were  selected  as  the  abscissa  and  ordinate 
respectively  to  provide  a  more  conventional  timeline  view  of 
activities.  The  echelon  dimension  was  collapsed,  and  only 
"high-level"  activities  associated  with  the  pre-invasion 
were  included  (historical  information  available  makes  the 
breakout  of  echelons  difficult  at  best) .  Additional 
information  describing  each  activity  is  provided  to  the 
right  of  the  matrix.  The  timeline  highlights  the  fact  that 
many  of  the  critical  activities  of  pre-invasion  were 
concentrated  in  time,  in  the  hours  and  even  minutes, 
preceding  the  Marine  landing.  It  also  shows  that  maneuver, 
air  defense  artillery,  and  tactical  air  functional  elements 
were  all  acting  almost  simultaneously  during  landing 
operations. 

The  third,  and  final  step  taken,  was  to  represent 
graphically  information  about  required  synchronization  in 
the  Inchon  landing  across  functional  areas.  For  this 
example,  it  was  assumed  that  the  purpose  of  each  activity 
and  the  interrelationships  between  activities  could  be 
inferred  with  reasonable  accuracy  from  historical  accounts 
of  the  operation.  Each  activity  was  then  examined  to 
determine  if  it  was  related  to  any  other  activity,  i.e., 
could  it  directly  influence  another  activity?  The 


6-7 


FUNCTIONAL  AREA 


8? 

a 


FIGURE  6-4.  Horizontal,  Vertical  And  Time  Dimensions 
Associated  With  Planning  And  Executing  A  Synchronized  Operation 
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interrelationships  and  coordination  information  were  then 
added  to  the  timeline,  as  shown  in  Figure  6-6. 

A  summary  of  the  type  information  developed  for  Figure 
6-6  is  as  follows: 

■  A  description  of  each  interrelated  activity  or 
event 

■  A  rationale  for  the  interrelationship 

■  The  relative  degree  of  importance  of  each 

activity  or  event 

■  The  desired  effect 

■  The  space  and  time  relationship  between  each 

interrelated  activity  or  event 

■  Critical  time/space  requirements  (e.g.  Aerial 

machine  gun  fire  must  commence  immediately  after 
landing  and  must  cover  area  50  yards  in  the  front 
of  Marines.) 

■  Required  coordination  between  functional 

elements  for  each  interrelated  activity  or 
event . 

Multi-Echelon  Synchronization 

Examining  the  activities  associated  with  the  Green 
Beach  invasion  at  Inchon  demonstrated  that  synchronization 
could  be  expressed  in  terms  of  functional  area  event 
timelines.  Time  and  event  relationships  on  the  battlefield 
that  might  typically  be  complex  and  difficult  to  visualize 
can  be  displayed  graphically  to  enable  clear  presentation  of 
the  coordination  required  to  successfully  perform  an 
operation.  The  success  of  this  application  of  the 
measurement  concept  prompted  the  research  team  to  further 
explore  the  application  of  the  synchronization  measurement 
concept  to  address  the  following  desired  capabilities: 

■  Compare  planned  operations  to  actual  operations  in 
a  way  that  deviations  from  the  plan  can  be  shown; 

■  Illustrate  the  synchronization  requirements  of 
more  than  one  echelon  of  command; 

■  Supplement  the  event  timelines  with  graphics 
illustrating  planned  and  actual  unit  locations. 
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FIGURE  6-6.  Interrelationship  Of  Inchon  Pre-Invasion  Activities 


To  address  these  issues.  Operation  Chromite  was 
revisited  and  the  simultaneous  invasions  at  Red  and  Blue 
Beaches  were  analyzed.  Timelines  depicting  the  plan  of 
operations  for  both  beaches  were  developed  at  corps  and 
battalion  level.  Actual  activities  and  events  were  then 
developed  on  a  separate  timeline  for  comparison.  Finally, 
the  use  of  maps  was  explored  to  depict  planned  and  actual 
locations  for  individual  units. 

Corps-Level  Operations 

The  complexity  of  the  Red  and  Blue  Beach  (RBB) 
invasions  was  similar  to  that  of  Green  Beach,  described 
above;  however,  the  number  of  activities  and  the  amount  of 
terrain  was  greater.  Consequently,  the  first  step  taken 
was  simply  to  catalog  each  activity  planned  and  any  other 
information  available  about  the  activity.  This  information 
is  summarized  in  Table  6-1.  Because  the  Red  and  Blue  Beach 
invasions  were  scheduled  to  begin  after  the  completion  of 
the  Green  Beach  invasion,  it  was  important  to  determine  if 
any  of  the  units  involved  with  the  earlier  assault  would  be 
included  in  the  later  assault.  As  could  be  expected,  the 
fire  support  and  tactical  air  elements  were  again  assigned 
to  bombard  the  invasion  areas.  In  addition,  an  engineering 
team  had  been  assigned  responsibility  for  securing  the 
Wolmi-do  Causeway  and  troops  at  the  northern  end  of  Wolmi-do 
were  expected  to  provide  covering  fire  to  the  RBB  assault 
force.  The  actual  invasion  force,  the  3rd  Battalion,  5th 
Marine  Division,  was  not  involved  at  Green  Beach. 

The  first  timeline  constructed  was  intended  to  portray 
the  RBB  invasion  as  it  was  planned  for  each  beach  and  as 
shown  in  Figure  6-7.  As  the  timeline  was  being  constructed 
it  became  apparent  that  most  planned  times  for  activities 
were  not  available.  Actual  times  for  activities,  where 
available,  were  used  instead.  As  shown,  four  functional 
areas  were  employed:  engineering  (ENGR) ,  fire  support  (FS) , 
tactical  air  (TAC  AIR) ,  and  maneuver  (MAN) .  The  timeline 
shows  that  corps  assets  particularly  in  the  FS  and  TAC  AIR 
functional  areas  were  not  simply  divided  in  two  to  assault 
the  beaches  independently,  although  some  of  these  assets 
were  dedicated  to  individual  beaches. 

Figure  6-7  also  depicts  the  diversity  of  activities 
planned  to  occur  prior  to  and  during  the  landing  of  marine 
forces.  Note  that  more  activities  were  scheduled  for  Red 
Beach.  Although  the  reason  for  this  is  not  given  in  the 
historical  reference  material  used,  it  is  apparent  from 
descriptions  of  the  fighting  on  each  beach  that  Red  Beach 
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TABLE  8—1.  RED  AND  BLUE  BEACH 
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TABLE  8-1.  RED  AND  BLUE  BEACH 
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FIGURE  6-7.  Red  And  Blue  Beach  Invasion  Timeline*  By  Beach 


was  more  heavily  fortified  by  NKPA  forces.  Advance 
knowledge  of  this  may  have  accounted  for  a  larger  scale 
assault  on  Red  Beach. 

Having  identified  the  activities  associated  with  each 
invasion  and  respective  functional  areas,  activities  were 
rearranged  on  the  timeline  so  as  to  be  arrayed  by  functional 
area  (Figure  6-8).  A  new  activity  was  added,  landing  craft 
vehicles  and  personnel  (LCVPs)  to  Beach,  at  the  beginning  of 
each  of  the  Red  and  Blue  Beach  Assaults.  Some  activities 
which  were  similar  enough  in  time  and  in  the  same  functional 
area  were  collapsed  into  one  activity  to  make  the  timeline 
easier  to  read.  The  Activity/Event  Reference,  located  on 
the  right,  references  each  activity  in  the  Activity  Summary 
presented  in  Table  6-1.  This  column  also  shows  where 
activities  were  collapsed,  e.g.,  4a-b  means  two  activities 
are  being  represented  by  Artillery  Fire  line.  As  was  the 
case  with  the  Green  Beach  operation,  the  Red  and  Blue  Beach 
invasion  had  a  great  number  of  activities  concentrated  in 
the  hours  and  minutes  before  the  marines  were  landed. 
MacArthur  reportedly  had  ordered  that  the  success  of  Green 
Beach  be  evaluated  before  commencing  any  further  invasion 
activities.  The  similarity  between  the  invasion  timelines 
reflects  the  success  of  the  Green  Beach  operation,  and 
MacArthur 's  desire  to  repeat  that  success. 

The  final  step  in  depicting  the  Red  and  Blue  Beach 
invasion  plan  was  to  show  the  degree  of  synchronization 
involved  across  functional  areas.  Figure  6-9  shows  six  key 
interrelationships  which  can  be  inferred  between  activities, 
five  of  which  involve  two  functional  areas. 

A  summary  of  the  information  developed  for  each 
interrelationship  is  provided  below.  Also  of  note  is  the 
degree  to  which  the  operation  was  keyed  to  the  maneuver 
function.  Every  activity  of  the  Red  and  Blue  Beach  invasion 
appears  to  have  been  in  some  way  related  to  preparing  for  or 
facilitating  the  maneuver  element's  activities  on  the  beach 
and  further  inland. 

Interrelationship  1  Minesweep  of  the  channels  must 
proceed  the  departure  of  the  LCVPs  to  Red  Beach.  Minesweep 
completion  and  location  are  critical  to  assure  the  safe 
approach  of  the  marines.  Time  is  criticr*  only  insofar  as 
the  minesweeping  activity  must  be  completed  by  1645. 
Coordination  is  not  required  unless  the  operation  plan  is 
changed. 

Interrelationship  2  Covering  fire  to  Red  Beach  and 
aerial  strafing  of  Red  and  Blue  Beaches  must  precede  the 
marine  landing  in  order  to  pin  down  the  NKPA  and  prevent 
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FIGURE  6-8.  Red  And  Blue  Beach  Invasion  Timeline:  By  Functional  Area 
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FIGURE  6-9.  Red  And  Blue  Beach  Invasion  Timeline:  Activity  Inter-relationships 


fire  on  the  marines.  Time  and  location  are  critical:  both 
activities  must  commence  immediately  after  the  marine 
landing  (1730)  and  must  cover  the  area  50  yards  in  front  of 
the  landing  force.  Coordination  may  be  required  between 
the  company  commanders  (Co  Cmdr)  and  the  Cmdr  of  the  TAC  AIR 
element . 

Interrelationship  3  Artillery  fire  and  rocket  fire  to 
Red  and  Blue  Beaches  prior  to  the  marine  landing  is  intended 
to  lessen  enemy  resistance.  Artillery  fire  time  and 
location  are  not  critical;  it  must  be  in  the  general  Red  and 
Blue  Beach  areas  and  must  precede  the  landing  force.  Rocket 
fire  time  and  location  are  important.  It  must  commence 
immediately  after  cessation  of  artillery  fire  and  end  when 
the  first  wave  of  marines  hit  the  seawall.  Rocket  fire 
should  occur  in  the  area  near,  but  east  of  marines  on  the 
seawall.  Coordination  may  be  required  between  the  Co  Cmdrs 
and  the  Cmdr  of  the  landing  ships,  medium  (rocket)  (LSMR) . 

Interrelationship  4  The  Wolmi-do  Causeway  should  be 
cleared  of  the  enemy  and  obstacles  prior  to  the  Red  Beach 
landing  to  enable  elements  of  the  3rd  Battalion  (BN) ,  5th 
Marines  to  seal  off  Wolmi-do  as  an  enemy  retreat  and  later 
to  provide  covering  fire  to  the  advancing  marines  on  Red 
Beach.  Timing  is  not  critical  unless  the  causeway  is  not 
secured  prior  to  the  start  of  the  Red  Beach  assault. 
Location  is  important;  the  entire  causeway  must  be  cleared. 
Coordination  is  not  required. 

Interrelationship  5  Aerial  bombardment  of  Red  and  Blue 
Beaches  prior  to  and  during  assaults  are  required  to  prevent 
enemy  reinforcement  and  to  disrupt  local  enemy  movement. 
Location  is  situation  dependent,  and  timing  is  not  critical 
since  the  activity  is  planned  to  occur  continuously  before 
and  during  the  marine  assault.  Coordination  may  be  required 
between  the  BN  Cmdrs  and  the  Fast  Carriers  77  Cmdr. 

Interrelationship  6  Continuous  aerial  reconnaissance 
and  bombardment  of  Red  and  Blue  Beaches  prior  to  and  during 
assaults  prevents  enemy  reinforcement  and  disrupts  local 
enemy  movement.  Location  is  situation  dependent  and  timing 
is  not  critical  since  the  activity  is  planned  to  occur 
before  and  during  the  marine  assault.  Coordination  may  be 
required  between  BN  Cmdrs  and  Task  Force  77  Cmdr. 

Comparing  corps-level  planned  operations  with  actual 
execution  of  events  proved  to  be  somewhat  difficult  for 
Operation  Chromite  particularly  due  to  the  lack  of  available 
information  on  planned  activity  times  which  would  normally 
be  available  in  the  corps  and  division  operations  plans. 
The  lack  of  information  may  also  be  due  in  part  to  the 
flexibility  built  into  the  plan.  For  example,  the  only 
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reported  start  time  for  any  of  the  invasion  activities  was 
H-hour,  the  landing  of  the  marine  force.  The  start  times 
for  other  activities  were  probably  keyed  to  H-hour,  though 
this  is  not  known  for  certain.  Also,  activity  end  times 
appear  to  have  been  somewhat  open-ended.  These  factors  make 
it  difficult  to  determine  if  any  activities  were  not 
completed  on  time.  Keeping  in  mind  that  Operation  Chromite 
and  in  particular  the  Red  and  Blue  Beach  invasions  were 
successful,  suggests  that  deviations  from  the  plan  were 
minor  and  may  only  be  found  at  lower  echelons. 

Battalion-Level  Operations 

Battalion-level  operations  were  investigated  in  an 
attempt  to  identify  events  that  did  not  go  as  planned  during 
Operation  Chromite,  and  to  look  at  synchronization  across  a 
second  echelon  of  command.  The  battalions  responsible  for 
the  invasion  of  Red  Beach  were  selected  because  Red  Beach 
operations,  though  successful,  did  not  go  exactly  as 
planned.  There  were  delays  and  some  subsequent  difficulties 
with  companies  arriving  at  their  assigned  locations.  The 
timelines  and  maps  developed  for  battalion  operations  were 
designed  to  illustrate  which  activities  did  not  go  as 
planned,  and  where  the  deviations  occurred  in  time  and 
space. 


The  5th  Marine  Regiment,  1st  and  2nd  Battalions,  were 
assigned  responsibility  for  seizing  and  securing  Red  Beach. 
The  3rd  Battalion  had  already  succeeded  in  taking  Green 
Beach,  and  this  area  was  secure  when  bombardment  of  the  Red 
and  Blue  Beaches  began.  The  organization  and  objectives  of 
the  six  companies  that  invaded  Red  Beach  are  shown  in  Figure 
6-10.  The  1st  Battalion  was  ordered  to  assault  the  northern 
area  of  the  beach  where  resistance  was  expected  primarily  at 
the  Asahi  Brewery  and  at  Cemetery  Hill.  The  1st  Battalion 
was  also  expected  to  join  flanks  with  2nd  Battalion  to 
perform  a  joint  assault  on  Observatory  Hill.  The  2nd 
Battalion  was  assigned  the  southern  portion  of  Red  Beach  to 
include  British  Consulate  Hill,  Wolmi-do  Causeway,  the  Tidal 
Basin,  and  the  southern  half  of  Observatory  Hill  (See  map  at 
Figure  6-11) . 

Invasion  of  Red  Beach  was  scheduled  for  1730  on  15 
September  1950.  The  invasion  force  would  arrive  in  waves, 
with  A  and  E  Companies  leading.  The  remaining  companies 
would  follow,  although  their  planned  time  of  arrival  is 
unknown.  The  operation  was  expected  to  commence  with 
simultaneous  assaults  on  Cemetery  Hill  and  British  Consulate 
Hill.  The  companies  arriving  in  the  second  wave  would  then 
join  forces  to  assault  Observatory  Hill,  which  was  expected 
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UNIT 

1st  Battalion 

A  Company 
B  Company 
C  Company 
2nd  Battalion 

D  Company 
E  Company 

F  Company 


OBJECTIVE 

Seize  and  Secure  area  3DD0  yards  long,  1DDD 
yards  deep  from  Cemetery  Hill  to  Inner  Tidal 
Basin  and  including  Observatory  Hill. 

Seize  and  Secure  Cemetery  Hill. 

Seize  and  Secure  No.  half  of  Observatory  Hill. 

Seize  and  Secure  Observatory  Hill. 

Seize  and  Secure  Southern  area  to  include 
British  Consulate  Hill  and  So.  half  of 
Observatory  Hill. 

Dislodge  and  Capture  enemy  on  Observatory  Hill. 

Sieze  and  Secure  British  Consulate  Hill  and  So. 
Observatory  Hill. 

Secure  Wolmi— do  Causeway  and  Tidal  Basin. 


FIGURE  6-10.  Red  Beach  Organizations  And  Objectives 
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FIGURE  €-11.  Red  Beach  Invasion  Map:  Planned 
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to  produce  some  fierce  fighting.  Figure  6-12  shows  the 
timeline  of  operations  for  each  company  and  the 
interrelationships  between  company  activities. 

The  first  interrelationship  is  fairly  straightforward. 
Obviously  the  timing  and  location  of  the  LCVP  approach  is 
critical  to  the  marine  forces  landing  at  the  right  place  and 
right  time.  Interrelationship  2  depicts  the  requirement  for 
Companies  B,  C,  D,  and  E  to  close  battalion  flanks  and  make 
a  coordinated  attack  on  Observatory  Hill.  However,  there 
are  actually  six  time  and  location  interrelationships 
between  these  activities: 

1.  B  Company  assault  is  related  to  C  Company  assault 

2.  B  Company  assault  is  related  to  D  Company  assault 

3.  B  Company  assault  is  related  to  E  Company  assault 

4.  C  Company  assault  is  related  to  D  Company  assault 

5.  C  Company  assault  is  related  to  E  Company  assault 

6.  D  Company  assault  is  related  to  E  Company  assault 

From  accounts  of  the  operation,  it  is  not  clear  if 
there  was  any  provision  for  coordination  between  companies 
once  the  assault  was  under  way.  Considering  the  number  of 
elements  involved  and  the  timing  necessary  to  coordinate  the 
attack  on  Observatory  Hill,  it  would  be  very  difficult  to 
execute  the  attack  as  planned  should  any  of  the  individual 
elements  fail  to  arrive  at  the  right  place  and  time.  This 
later  proved  to  be  a  problem. 

The  first  problem  occurred  when  the  1st  and  3rd 
Platoons  of  A  Company  were  delayed  at  the  Red  Beach  seawall 
because  they  found  themselves  pinned  down  by  enemy  fire. 
Also,  one  of  the  landing  craft  vehicles  and  personnel  got 
hung  up  just  off  the  seawall  when  it  had  engine  failure. 
These  delays  did  not  disrupt  operations  significantly,  as 
2nd  Platoon  was  able  to  move  out  from  the  seawall  and 
quickly  assaulted  and  secured  Cemetery  Hill.  E  Company, 
also  in  the  first  wave,  did  not  encounter  any  delays  and  was 
able  to  move  out  as  a  unit  and  secure  British  Consulate 
Hill.  The  second  wave  of  marines  did  not  have  the  same 
luck.  The  commander  of  C  Company,  Lt  Pederson,  and  his 
troops  were  diverted  to  assist  the  helpless  LCVP.  As  a 
consequence  parts  of  C  Company  were  deposited  at  the  wrong 
area  at  the  wrong  time.  D  Company  also  experienced  similar 
problems  at  landing  which  meant  that  two  of  the  three 
companies  assigned  to  take  Observatory  Hill  were  in  a 
somewhat  confused  state  shortly  after  their  arrival  at  Red 
Beach. 
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FIGURE  6-12.  Red  Beach  Activity  Timeline  And  Interrelationships 


Once  all  the  companies  had  arrived,  more  changes  to  the 
planned  assault  occurred.  Because  members  of  C  and  D 
Companies  had  difficulty  finding  and  joining  their 
respective  platoons,  the  1st  and  2nd  Battalions  could  not 
properly  join  flanks  prior  to  approaching  Observatory  Hill. 
Their  assault  was  not  proceeding  well  and  F  Company  was  re¬ 
assigned  to  Observatory  Hill  to  provide  some  additional 
firepower.  This  proved  to  be  effective  in  completing  the 
immediate  objective,  but  did  not  enable  the  Causeway  or 
Tidal  Basin  to  be  secured.  For  unexplained  reasons,  some 
landing  ship,  tanks  (LSTs)  were  started  up  on  the  beach 
about  the  same  time  C  and  D  Companies  were  experiencing 
difficulties.  The  movement  was  apparently  intended  to  draw 
fire  from  the  NKPA  forces  on  Observatory  Hill  in  order  to 
give  away  their  location.  The  maneuver  worked,  but  the  LSTs 
did  not  stop  at  this  point.  They  then  returned  fire  on  the 
enemy,  but  also  directed  fire  at  Cemetery  Hill  and  other 
apparently  randomly  selected  targets.  The  other  result  of 
this  maneuver  was  the  loss  of  1  marine  and  23  others 
wounded.  The  Red  Beach  invasion  timeline,  as  these 
activities  actually  occurred,  is  shown  in  Figure  6-13. 
Figure  6-14  shows  the  deviation  in  location  of  F  Company  as 
a  result  of  their  re-assignment  to  Observatory  Hill.  Also 
shown  are  the  planned  and  actual  locations  for  the  1st 
Battalion,  1st  Regiment  assigned  to  the  Blue  Beach  invasion. 
When  luck  is  mentioned  with  regard  to  MacArthur's  operation, 
these  two  units  and  their  successes  may  be  considered  a  part 
of  that  luck.  When  F  Company  vacated  the  tidal  basin  area, 
they  left  the  right  flank  of  the  2nd  Battalion  exposed. 
During  the  Blue  Beach  landing,  however,  the  1st  Battalion, 
1st  Regiment  was  mistakenly  taken  to  the  area  southwest  of 
the  tidal  basin.  They  should  have  landed  2  miles  farther 
south.  The  1st  Battalion  was  a  reserve  force  during  the 
Blue  Beach  assault,  and  consequently  their  absence  was  not 
immediately  problematic.  When  they  arrived  at  the  tidal 
basin  area  the  battalion  commander  quickly  evaluated  the 
situation  and  was  able  to  secure  the  area  before  moving 
south  to  join  the  Blue  Beach  forces.  The  result  was  that 
all  objectives  were  taken. 

CpnclMsion? 

The  graphic  representations  of  the  Inchon  landing 
provide  evidence  that  synchronization  in  an  operation  can  be 
expressed  in  terms  of  functional  areas  and  time.  A  timeline 
of  activities  and  events  can  be  used  to  construct  a  model 
which  provides  a  picture  of  how  an  operation  unfolded.  The 
information  in  the  timeline  is  particularly  useful  in 
summarizing  key  battlefield  events  and  interrelated 
activities.  A  similar  timeline,  depicting  planned 
activities  and  events  provides  comparative  information  to 
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Figure  6-13.  Red  Beach  Invasion  Timeline:  Actual 
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FIGURE  €-14.  Red  Beach  Invasion  Map: 
Planned  And  Actual  Location*  Of  F  Company 
1st  Battalion 
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identify  where  an  operation  deviated  from  its  plan.  When 
presented  as  part  of  an  after-action  review  following  an 
exercise,  these  timelines  will  enable  a  commander  and  his 
staff  to  quickly  see  which  elements  of  the  force  were  able 
to  execute  on  time  and  which  elements  had  problems  meeting 
the  timetable. 

Functional  areas  in  which  problems  in  execution  were 
demonstrated  can  be  examined  in  greater  detail  by  generating 
micro-timelines  to  summarize  the  activities  of  each  func¬ 
tional  area.  The  Red  Beach  example  demonstrated  the  use  of 
such  multi-echelon  timelines  and  situation  maps.  The  Red 
Beach  example  also  demonstrated  that  both  time  and  space  are 
important  dimensions  in  achieving  synchronized  operations. 
In  the  case  of  C  and  D  Companies'  late  arrivals,  the  two 
dimensions  were  seen  to  interact  (e.g.  the  late  arrival  of  C 
Company  prevented  them  from  massing  at  the  appropriate 
location) .  Unplanned  activities  such  as  the  LST  maneuver 
can  be  shown  and  in  fact  could  be  related  to  later  positive 
or  negative  battle  activities  and  outcomes,  activity  or 
group  of  activities. 

The  application  of  the  timelines  to  the  Inchon  Landing 
example  confirmed  that  the  graphic  representation  of 
information  describing  the  interrelatedness  of  battlefield 
activities  and  events  shows  promise  for  use  in  developing 
measures  of  synchronization.  For  example,  when  a  timeline 
of  interrelated  activities  across  functional  areas  is 
constructed,  as  was  done  for  the  Green,  Red,  and  Blue  Beach 
landings,  one  has  taken  the  first  step  in  identifying  how 
those  activities  must  be  coordinated  among  functional 
elements.  This  is  an  essential  element  in  a  corps-level 
performance  measurement  system  since  the  information  from 
the  timelines  can  be  used  to  indicate  where  particular  staff 
actions  must  occur  at  key  points  in  time.  Again,  micro¬ 
timelines  at  various  echelons  will  also  provide  the 
structure  to  track  such  things  as:  interfaces  between 
functional  elements  and  echelons;  the  objective  of  each 
activity;  and  the  precision  of  execution  in  time  and  space. 
The  timeline  can  also  be  used  as  an  audit  trail  to  identify 
activities  that  did  or  did  not  occur  according  to  plan,  and 
as  a  vehicle  for  developing  feedback  to  the  commander  and 
his  staff.  Parallel  timelines  which  relate  staff  activities 
over  time  to  battlefield  events,  are  additional  tools  to  be 
used  in  providing  feedback  data  to  the  commander. 

Collectively,  the  tools  described  above  represent  a 
potential  structure  for  development  of  measures  of  corps 
performance.  The  time  dimension  and  specification  of 
discrete  events  and  linkages  between  functional  areas  at 
particular  points  in  time  provide  the  starting  points  for 
development  of  quantitative  measures  of  synchronization.  As 
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will  be  discussed  in  sections  7  and  8,  such  measures  might 
include  variables  such  as  frequency  counts  of  the  number  of 
cross-functional  connections  required  and/or  achieved  in  an 
operation  or  the  average  deviation  betveen  the  time  an  event 
was  planned  and  when  it  was  actually  executed. 

The  application  of  the  synchronization  measurement 
concept  to  the  Inchon  Landing  provides  support  for  the 
viability  and  utility  of  the  measurement  concept.  It  should 
be  recognized,  however,  that  the  information  available  for 
applying  the  measurement  concept  was  extremely  limited.  The 
measurement  of  corps  performance  will  require  the  collection 
of  a  variety  of  performance  measures  in  addition  to  the 
development  of  the  timelines  illustrated  in  the  Inchon 
Landing  example.  Section  7  outlines  the  structure  of  the 
entire  corps  performance  measurement  concept  and  Section  8 
will  then  describe  the  data  collection  required  to  implement 
such  a  performance  measurement  system. 
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Introduction 


SECTION  7 

THE  PERFORMANCE  MEASUREMENT  CONCEPT 


The  battlefield  event  timelines  used  in  the  discussion 
of  the  Inchon  Landing  Operation  in  the  previous  section, 
represent  specific  examples  of  a  measurement  tool  which  can 
be  used  to  begin  to  quantify  the  construct  of 
synchronization  in  terms  of  discrete  events  and  time.  While 
the  events  and  time  scales  must  necessarily  change  from  one 
corps  operation  to  the  next,  the  general  structure  of  the 
timelines  will  remain  the  same.  The  performance  measurement 
concept  described  in  this  section  of  the  resort  represents  a 
more  comprehensive  set  of  measurement  tools  which  may  be 
used  across  a  number  of  different  exercises  to  assess  the 
performance  of  Army  corps. 

The  Measurement  Challenge 

The  development  a  set  of  highly  specific  measurement 
instruments  for  use  in  all  corps  level  operations  is 
probably  impossible.  At  the  division  and  corps  levels, 
operations  are  extremely  complex  and  the  principle  of 
equi finality  applies,  i.e.  there  are  multiple  ways  to 
achieve  any  given  outcome  or  objective.  Thus,  a  measurement 
approach  based  on  specification  of  mission  specific  tasks  or 
functions  to  measure  synchronization  has  little  probability 
of  succeeding.  As  illustrated  in  the  Inchon  Landing 
example,  the  operational  definition  of  synchronization  may 
only  be  achieved  within  the  context  of  a  particular  military 
operation  or  exercise.  On  the  other  hand,  a  measurement 
strategy,  structure  of  the  measurement  process,  and 
methodology  for  development  of  operational  definitions  of 
synchronization  can  be  constructed  for  application  within 
the  context  of  any  corps-level  operation. 

In  this  section  of  the  report,  the  authors  will  present 
a  general  measurement  strategy  and  the  basic  structure  of  a 
measurement  system  for  generating  corps-level  performance 
measures  within  the  context  of  joint  operations  exercises 
conducted  by  the  US  Readiness  Command  (REDCOM)  using  the 
Joint  Exercise  Support  System  (JESS)  and  the  Joint  Theater 
Level  Simulation  (JTLS)  as  exercise  drivers.  Section  8  will 
present  a  detailed  discussion  of  the  data  collection  plan 
for  operationalizing  the  Measurement  system  within  the 
REDCOM  context.  Those  readers  unfamiliar  with  the  joint 
exercise  program  conducted  by  REDCOM  should  review  the 
description  of  the  exercise  process  and  the  major  components 
of  the  exercise  system  presented  in  Appendix  C. 
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Measurement  Strategy 


The  measurement  strategy  adopted  for  development  of 
corps  level  measure's  of  performance  is  concept-based  rather 
than  data-driven.  This  means  that  the  starting  point  for 
the  development  of  the  measurement  methodology  was  the 
identification  of  a  performance  construct  or  concept  rather 
than  a  delineation  of  data  which  might  be  obtained  in  the 
context  of  the  REDCOM  simulation  as  it  currently  exists. 
The  concept-based  measurement  development  process  ultimately 
requires  the  development  of  measurement  procedures  to 
capture  data  within  a  particular  context.  However,  it 
provides  a  general  measurement  framework  which  can  be 
applied  across  a  variety  of  situations.  The  concept-based 
measurement  strategy  is  also  less  likely  to  result  in  the 
development  of  procedure-oriented  task,  condition,  and 
standard  measurement  instruments.  Such  "task  checklists" 
are  usually  developed  as  a  result  of  a  job  analytic  approach 
to  performance  measurement  that  focuses  on  identification  of 
observable  behaviors  which  occur  during  performance  of  tasks 
associated  with  particular  organizational  objectives.  Such 
bottom-up  approaches  may  be  appropriate  at  the  small  unit 
level  for  the  development  of  performance  measures  in  fairly 
standardized  battle  drills  and/or  routine  staff  procedures. 
They  are  not,  however,  appropriate  for  application  to  the 
operation  of  complex  organizations  such  as  found  at  the 
corps  and  EAC  levels. 

The  concept  which  forms  the  basis  for  the  measurement 
strategy  adopted  in  the  current  effort  is,  obviously,  the 
concept  of  synchronization.  A  legitimate  question  which 
must  be  addressed  is  why  synchronization?  A  major  pitfall  in 
the  use  of  a  concept-based  measurement  strategy  is  that  it 
may  easily  become  a  conceptual  measurement  strategy  which  is 
not  reality-based.  That  is,  the  researcher  may  develop  a 
measurement  strategy  which  is  derived  directly  from  a 
particular  organizational  or  information  processing  model. 
The  measurement  procedures  developed  in  this  situation  are 
likely  to  have  meaning  to  researchers  familiar  with  the 
theory  or  conceptual  model,  but  may  provide  data  which  are 
of  little  value  to  the  military  audience  for  which  the 
measurement  system  is  designed. 

If,  on  the  other  hand,  the  concept  driving  the 
development  of  the  measurement  system  is  drawn  directly  from 
relevant  military  doctrine,  the  system  is  likely  to  provide 
meaningful  feedback  to  the  military  audience  —  assuming  of 
course  that  the  concept  is  operationally  defined  in  terms  of 
accurate  and  meaningful  measurement  procedures.  The  concept 
of  synchronization  is  one  of  the  four  tenets  which  underlie 
basic  AirLand  Battle  doctrine  as  presented  in  FM  100-5. 
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Synchronization  is  perhaps  the  least  well  defined  and 
most  frequently  discussed  of  the  tenets  underlying  AirLand 
Battle  doctrine.  The  frequency  with  which  synchronization 
is  discussed  in  articles  appearing  in  professional  military 
journals  such  as  Military  Review  reflects  both  the 
recognition  of  the  importance  of  the  concept  and  the 
complexity  and  difficulties  surrounding  its  understanding. 
Thus,  any  progress  made  in  the  current  research  effort  which 
aids  in  operationally  defining  synchronization  will 
represent  a  major  contribution  to  the  literature. 

A  second  reason  for  selecting  synchronization  as  the 
concept  driving  the  development  of  corps  level  measures  of 
performance  is  that  synchronization  is  perhaps  the  most 
relevant  of  the  four  AirLand  Battle  tenets  for  corps  level 
operations.  In  effect,  the  decision  has  been  made  to  define 
corps  level  performance  primarily  in  terms  of  the 
performance  of  the  corps  commander  and  his  staff.  As  noted 
earlier  in  the  discussion  of  nested  unit  effects,  it  is 
extremely  difficult  to  use  bottom-line  combat  outcomes  to 
represent  "corps  level"  performance  since  they  also 
represent  the  performance  of  all  units  between  the  corps  and 
squad  or  team  levels. 

The  review  of  AirLand  Battle  doctrine  regarding 
operational  levels  of  war  and  examination  of  various 
documents  related  to  corps  organizations  and  corps  staff 
functions  suggests  that  the  critical  role  played  by  the 
corps  is  that  of  tactical  battle  management.  Echelons  above 
corps  also  have  operational  roles  but  often  are  more 
concerned  with  strategic  implications  of  actions  rather  than 
management  of  the  tactical  aspects  of  the  battle.  The 
echelons  below  corps  represent  maneuver  elements  which 
function  almost  entirely  at  the  tactical  level.  While 
commanders  and  staffs  at  the  division  level  and  below  must 
manage  battle  assets,  they  are  primarily  fighting  the  battle 
rather  than  managing  the  battle.  One  might,  in  fact, 
suggest  that  the  primary  mission  of  the  corps  is  to 
synchronize  the  operations  at  division  level  and  below  and 
to  provide  the  primary  focal  point  for  synchronization  of 
joint  operations  between  ground,  air,  and  naval  forces. 

Another  aspect  of  the  measurement  strategy  adopted  in 
the  current  effort  is  that  it  will  be  based  on  the  use  of  a 
probe  methodology.  A  probe  is  a  designated  event  or 
sequence  of  events  which  should  stimulate  the  occurrence  of 
a  particular  set  of  actions  or  behaviors.  These  behaviors 
or  actions  are  designated  as  targets  on  which  observers 
focus  their  attention  after  the  probe  has  been  implemented 
during  the  exercise  or  simulation.  This  methodology  is  a  an 
essential  part  of  the  measurement  strategy.  Without  the  use 
of  probes,  the  task  of  the  observers  or  data  collectors  is 
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unmanageable  within  an  exercise  as  complex  as  those 
conducted  at  the  corps  level. 

The  authors  recognize  that  the  use  of  a  concept-based 
measurement  strategy  also  imposes  certain  limitations.  By 
focusing  on  the  construct  of  synchronization,  other 
performance  factors  such  as  the  commander's  conceptual  frame 
of  reference  will  receive  less  attention  in  terms  of 
measurement.  As  will  be  seen  later  in  this  section, 
however,  numerous  types  of  data  will  be  generated  as  part  of 
the  performance  measurement  system. 

Structure  of  the  Measurement  Process 

The  basic  structure  of  the  measurement  process 
developed  in  the  current  research  effort  will  be  determined 
by  a  number  of  factors  including: 

■  The  purpose  or  objective  of  the  measurement  system 
(training) 

■  The  performance  concept  to  be  operationalized 

( synchronization ) 

■  The  level  of  the  organization  being  assessed 

(corps) 

■  The  general  context  in  which  the  performance 

measurement  will  occur  (corps  level  simulation 
exercises) 

■  The  available  data  collection  technology  (computer 
simulation  data,  video-tapes,  observations) 

As  noted  earlier,  the  primary  purpose  for  developing 
the  performance  measurement  system  in  the  current  research 
effort  is  to  provide  training  feedback  to  corps  commanders 
and  their  staffs.  The  general  context,  level  of  the 
organization,  and  synchronization  concept  have  all  been 
discussed  previously.  As  noted  above,  the  strategy  used  in 
the  current  measurement  development  process  is  concept-based 
rather  than  data  driven.  To  remain  reality-based,  however, 
the  structure  of  the  performance  measurement  system  must 
reflect  the  range  of  data  collection  technologies  which  are 
available.  In  the  current  effort,  it  is  assumed  that 
essentially  four  types  of  data  gathered  through  three 
technologies  are  available.  First,  battle  outcomes  and  a 
limited  amount  of  battlefield  action/event  data  can  be 
obtained  directly  from  the  data  stored  in  the  memory  of  the 
computer  simulation  used  as  the  exercise  driver  by  REDCOM. 
Second,  records  of  player  actions,  orders,  etc.,  can  be 
obtained  through  the  use  of  videotapes  and  records  made  by 
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observers.  Finally,  observations  of  critical  information 
processing  behaviors  and  the  tracking  of  noncomputerized 
documents/products,  etc.  can  be  observed/ inferred  and 
recorded  by  observers. 

Using  a  broadly  defined  concept  of  synchronization  as 
the  starting  point  and  assuming  a  REDCOM  exercise  context  as 
the  site  of  implementation  of  the  measurement  system,  a 
basic  structure  for  the  measurement  system  is  outlined 
below.  It  should  be  stressed  £hat  this  basic  structure  is 
tentative  in  nature  and  may  be  modified  significantly  after 
its  feasibility  is  assessed  through  observations  at  a  REDCOM 
exercise. 

At  the  core  of  the  measurement  system  will  be  a 
structural  model  of  corps  actions  and  information  flow.  The 
model  will  be  three  dimensional  with  a  vertical  dimension 
representing  echelons  (EAC  through  Brigade  level)  and  a 
horizontal  dimension  representing  critical  functional  areas 
to  include  Air  Force  and  Naval  elements.  The  third 
dimension,  (depth)  represents  the  time  dimension.  This 
matrix  is  illustrated  in  Figure  6-4  in  Section  6.  Different 
cells  within  the  three-dimensional  matrix  will  be 
representative  of  particular  staff  and  command  elements 
which  are  considered  to  be  critical  players  in  the  corps 
operation.  Some  cells  at  various  echelons  may  be  empty 
cells.  Moving  through  the  time  dimension,  the  cells  will 
(represent  the  different  staff  and  command  elements  at 
various  stages  or  phases  of  the  operation.  The  time 
dimension  will  begin  at  the  start  of  the  planning  for  an 
operation  and  end  with  the  conclusion  of  the  operation. 
Thus,  the  time  dimension  for  a  REDCOM  operation  may  begin 
several  days,  weeks  or  even  months  prior  to  the  time  when 
the  simulation  driver  model  (JESS)  is  initialized  and  the 
exercise  ’'begins". 

The  three-dimensional  matrix  described  above  represents 
the  cornerstone  of  the  measurement  structure.  The  matrix 
provides  the  conceptual  basis  for  the  development  of  the 
battlefield  and  staff  action  event  timelines  described  in 
the  Inchon  Landing  example.  The  various  components  of  the 
measurement  system  are  designed  to  provide  data  required  to 
construct  a  series  of  such  event  timelines  which,  when 
combined  with  other  data,  can  be  analyzed  and  synthesized  to 
develop  a  description  of  corps  performance.  Before 
describing  the  various  components  of  the  measurement  system, 
it  will  be  beneficial  to  present  an  overview  of  the 
measurement  process. 
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overview  of  the  Measurement  Process 

The  corps  performance  measurement  process  is 
illustrated  graphically  in  Figure  7-1.  The  process  will 
begin  with  an  analysis  of  the  EAC  operations  plan.  From 
this  plan,  the  role  and  objectives  of  the  corps  operation 
will  be  derived.  A  matrix  will  be  constructed  which 
identifies  critical  events  and  times  for  activities  for 
which  the  corps  is  responsible.  The  elements  of  different 
echelons  of  the  corps  involved  in  the  activities  will  be 
identified  later  during  the  analysis  of  the  corps  operations 
plan.  Information  must  be  gathered  at  this  time  to  determine 
the  structure  of  the  corps,  organization  of  the  corps  staff, 
and  SOPs  of  the  corps  staff  actions.  This  information  is 
required  because  these  factors  vary  from  one  corps  to 
another.  Observers  will  then  focus  their  attention  on  the 
primary  staff  elements  of  the  corps  involved  in  development 
of  the  corps  operations  plan,  particularly  the  G2,  G3,  and 
G4  elements.  The  focus  of  these  observations  will  be  on 
information  processing  with  data  collection  including  time 
required  to  gather,  process,  and  disseminate  information  and 
orders. 

Once  the  corps  operations  order  is  developed,  this 
order  will  be  analyzed  to  construct  an  event  timeline  of  key 
battlefield  events  and  their  interrelationships.  This 
timeline  will  be  constructed  on  a  micro-computer  and  will  be 
used  to  determine  the  degree  of  synchronized  activity  called 
for  in  the  corps  operations  plan.  Using  this  battlefield 
event  timeline,  a  second  timeline  will  be  constructed  which 
identifies  critical  staff  events  which  must  occ'-r  in  order 
for  the  battlefield  events  to  occur  as  planned.  This  latter 
timeline  will  be  used  to  identify  critical  staff  elements, 
staff  integration  required  to  implement  the  operation, 
likely  information  flow  patterns  and  critical  information 
processing  nodes  for  the  operation.  Given  the  current 
structure  of  REDCOM  exercises,  all  of  the  above  processes 
would  be  completed  prior  to  the  actual  time  at  which  the 
exercise  is  scheduled  to  start.  Thus,  observations  of  staff 
planning  and  development  of  the  corps  operations  plan  would 
need  to  occur  at  the  players'  home-station  location.  No 
attempt  to  observe  such  corps  planning  activities  is 
currently  undertaken  by  REDCOM. 

In  addition  to  the  battlefield  event  timelines  developed  for 
the  Army  Corps  participating  in  the  REDCOM  exercise,  a 
parallel  battlefield  event  timeline  will  be  constructed  for 
the  Opposing  Force  (OPFOR)  operations  plan  for  the  exercise. 
This  timeline  will  identify  all  of  the  key  events  and 
interrelationships  which  are  contained  in  the  OPFOR  plan  of 
operations.  This  timeline  will  be  an  important  source  of 
information  used  for  feedback  to  the  corps. 
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ANALYZE  CORPS  OPERATIONS  PLAN 
DEVELOP  TIME  BY  EVENT  TIMELINES  AND 
STATEMENT  OF  COMMANDER'S  INTENT 


FIGURE  7-1.  Schematic  Of  Measurement  Process 


Based  on  the  corps  operations  plan  and  the  op  order 
delivered  by  the  corps  commander,  the  observation  team  must 
construct  and  verify  with  the  corps  commander  a  concise 
statement  of  the  commander's  concept  of  the  operation.  This 
concept  must  include  clear  statements  of  the  roles  of  the 
corps  elements  whose  actions  are  essential  to  the  successful 
execution  of  the  plan.  After  verification  of  the  plan, 
questions  must  be  developed  which  can  be  answered  by  key 
players  to  provide  an  objective  measure  of  the  degree  to 
which  they  understand  the  commander's  intent  and  their  role 
in  the  operation.  The  questions  may  be  presented  to  the 
selected  players  in  the  form  of  oral  questions  or  a  short 
questionnaire  administered  at  the  end  of  the  exercise. 

After  the  battlefield  event  and  staff  action  timelines 
have  been  constructed  for  both  the  Army  Corps  (Blue)  and 
OPFOR,  the  next  step  in  the  measurement  process  would  be  to 
identify  and  schedule  probe  events  for  implementation  during 
the  exercise.  Given  the  fact  that  the  exercises  are  largely 
free  play,  it  will  probably  be  impossible  to  specify  times 
at  which  probes  are  implemented.  Rather,  the  relative  order 
of  probe  implementation  and  various  probe  implementation 
contingency  plans  will  be  developed. 

Concomitant  with  the  development  of  the  probe  schedule 
will  be  the  identification  of  critical  observation  points 
during  various  stages  of  the  exercise.  These  observation 
points  will  represent  various  staff  elements  at  different 
echelons.  Observers  must  be  assigned  to  cover  these 
elements  during  certain  planned  corps  actions  and  when 
activities  stimulated  by  the  probes  are  anticipated. 
Specific  target  behaviors  to  be  observed  must  be  identified 
and  the  appropriate  observation  guides  assembled  for 
observers  manning  the  different  observation  sites.  The 
observation  guides  will  consist  primarily  of  observer 
prompts  and  data  recording  forms  which  will  be  loaded  into 
electronic  clipboards  that  allow  convenient  recording  of 
data.  After  observations  have  been  made  and  recorded,  the 
data  can  be  automatically  uploaded  into  the  same  micro¬ 
computers  used  to  construct  the  event  by  time  matrices. 

The  appropriate  triggers  for  automated  data  collection 
must  also  be  programmed  into  the  exercise  driver  model  prior 
to  the  exercise.  Just  as  the  probe  schedule  can  be 
developed  only  loosely,  the  trigger  programs  for  automatic 
data  collection  must  be  developed  to  allow  manual 
initialization  of  the  programs.  Prior  to  the  exercise  it 
will  be  possible  to  program  the  identification  of  the  types 
of  data  and  elements  (work  stations)  from  which  data  are  to 
be  collected  during  various  phases  of  the  exercise  or 
following  the  implementation  of  a  particular  probe. 
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Once  the  exercise  actually  begins,  the  measurement 
rystem  will  function  as  follows.  At  predetermined  time 
intervals,  during  set  periods  of  time  following  the 
insertion  of  a  probe,  and  at  times  a  senior  observer  makes 
special  requests,  the  automatic  data  collection  programs 
will  record  the  following  types  of  information  from  the 
exercise  driver  model  (currently  JESS) : 

1.  Beginning  time  for  data  collection 

2.  Identification  of  BLUEFOR  and  OPFOR  units  in  area 
of  interest 

3 .  Unit  status  of  BLUEFOR  and  OPFOR  elements 

(personnel  strength,  number  and  type  of  weapons 
systems,  etc.) 

4.  Record  of  dispositions  and  movement  of  each 

BLUEFOR  and  OPFOR  unit 

5.  Casualty  and  BDA  reports  if  units  are  engaged  in 
conflict 

6.  Listing  of  orders,  messages,  movement  commands, 

etc.  relevant  to  units  under  observation 

7.  Ending  time  for  data  collection 

While  the  data  elements  listed  above  are  recorded 
automatically,  observers  will  be  using  electronic  clipboards 
to  record  data  from  relevant  staff  elements  involved  in  the 
actions  of  interest.  The  majority  of  the  observations 
recorded  will  focus  on  actions  of  commanders  and  information 
processing  actions  of  the  staff.  At  regularly  scheduled 
time  intervals,  all  data  recorded  by  observers  on  electronic 
clipboards  will  be  uploaded  on  the  micro-computer  for 
processing  and  analysis. 

At  appropriate  time  intervals,  printouts  of  the  data 
collected  automatically  on  JESS  will  be  examined  and  certain 
events  will  be  extracted  and  used  to  construct  another 
battlefield  event  timeline  of  the  corps  operation  as  it  is 
actually  implemented.  This  timeline  will  be  compared  to  the 
planned  operation  to  determine  MAJ  or  deviations  from  the 
plan  and  will  be  a  critical  part  of  the  feedback  system. 
Where  deviations  between  the  event  timelines  of  planned  and 
executed  events  occur,  the  deviations  will  be  noted  and 
relevant  data  on  related  staff  and  command  actions  collected 
by  observers  will  be  examined.  A  parallel  battlefield  event 
timeline  of  actual  OPFOR  events  will  also  be  constructed  and 
compared  to  planned  OPFOR  events. 
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At  the  end  of  the  exercise,  the  timelines  of 
battlefield  events  and  staff  actions  and  supporting  data 
gathered  automatically  and  through  observers  will  be 
compiled  to  provide  feedback  to  key  members  of  the  corps. 
This  processing  of  data  will  be  facilitated  by  the  computer- 
based  system  used  to  record  data  and  construct  the  event 
timelines.  The  content  of  the  feedback  will,  of  course, 
vary  from  exercise  to  exercise  depending  on  the  nature  of 
the  operation  and  the  manner  in  which  the  operation  is 
executed.  The  basic  structure  or  format  for  the  feedback 
system  will  remain  essentially  the  same,  however.  Two  basic 
types  of  feedback  will  be  given.  The  first  will  represent  a 
narrative  of  the  operation  with  feedback  using  the  BLUE  and 
OPFOR  timelines  illustrating  events  as  they  were  planned  and 
as  they  were  actually  executed.  Accompanying  this 
discussion  will  be  various  battle  outcome  data  for  the 
exercise  as  a  whole  and  for  selected  engagements  critical  to 
the  operation. 

After  discussion  of  what  happened  and  presentation  of 
data  relevant  to  the  level  of  performance  of  the  corps  as  a 
(whole,  the  feedback  will  shift  to  a  discussion  of  why  the 
corps  performed  as  it  did.  Feedback  presented  in  this 
portion  of  the  after  action  review  will  be  structured  in  a 
backward  audit  fashion.  The  feedback  will  focus  on  a 
limited  number  of  key  events  which  the  senior  observer 
identified  as  critical  to  synchronization  of  the  corps 
operations.  The  events  will  likely  be  related  to  probes 
which  were  inserted  into  the  exercise.  Starting  with  the 
outcome  of  these  events,  data  will  be  presented  to  examine 
the  sequence  of  battlefield  events  and  staff  actions  related 
to  the  outcomes.  Finally,  the  battlefield  events  and  staff 
actions  will  be  traced  back  to  the  probe  (Intel  information 
and/or  OPFOR  actions)  which  served  as  the  stimulus  for  the 
event. 

Elsstranis  Cllpfrpflrd  Technology 

In  order  to  implement  the  measurement  process  described 
above,  a  number  of  measurement  instruments  or  components  of 
the  system  must  be  developed.  The  electronic  clipboard 
technology  to  be  used  in  recording  observations  has  already 
been  developed  and  pilot-tested  by  ARI.  The  clipboards  make 
use  of  touch  sensitive  screens  and  menus  to  eliminate  the 
need  for  typing  of  information.  The  clipboards  also  provide 
prompts  and  instructions  for  the  observers  and  automatically 
time-tag  data  when  it  is  entered  into  the  machine.  The 
clipboards  presently  operate  for  approximately  eight  hours 
on  a  rechargeable  battery.  Data  recorded  on  the  clipboard 
can  be  uploaded  onto  an  IBM  PC  or  compatible  micro-computer. 
The  specific  observer  prompts  and  data  collection  menus  used 
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with  the  clipboards  can  be  downloaded  into  the  clipboard 
using  the  same  PC  system.  The  content  of  the  data 
collection  menus  and  observer  prompts  is  easily  modified 
using  word  processing  software  on  the  PC. 

P^??rva.t  iQE-Stilflsg 

The  development  of  the  content  of  the  observation 
guides  to  be  loaded  into  the  electronic  clipboards  will 
require  that  a  number  of  tasks  be  completed.  First, 
taxonomies  of  the  type  of  information  and/or  actions  to  be 
recorded  by  observers  must  be  constructed.  Information  will 
be  categorized  in  terms  of  dimensions  such  as  content 
(tactical,  logistical,  etc.),  relative  level  of  source 
(corps,  above  Cozps,  below  corps),  whether  the  information 
is  basically  an  input  or  output  of  the  corps  commander  and 
his  staff  (i.e.,  intelligence  information  vs.  a  FRAG  order), 
and/or  the  functional  area  through  which  information  would 
enter  or  leave  the  corps  (Intelligence,  Maneuver  elements, 
etc.)  These  different  types  of  information  represent 
potential  targets  on  which  observers  might  focus  their 
attention  after  the  occurrence  of  a  probe. 

In  addition  to  a  taxonomy  of  information  which  is 
defined  in  a  militarily  meaningful  manner,  the  observation 
guides  will  contain  a  component  which  identifies  potential 
information  processing  activities  which  may  be  of  interest. 
Such  processes  may  include  relaying  information, 
synthesizing  information,  analyzing  information,  trans¬ 
forming  information  (verbal  to  graphics,  etc.).  These 
processes  will  be  translated  from  their  conceptual 
information  processing  terminology  into  observable  military 
behaviors  which  might  be  displayed  by  commanders  and  their 
staffs.  This  component  of  the  measurement  system  will  be 
combined  with  the  information  taxonomy  to  develop  some  form 
of  structured  observation  guides  or  observation  framework 
for  observers  and  for  use  in  development  of  data  extraction 
programs  for  extracting  data  from  the  simulation  model 
itself. 

Another  element  of  the  performance  measurement  system 
related  to  the  observation  guides  will  be  the  observer 
training  component.  This  training  program  will  be  used  to 
familiarize  data  collectors  with  the  concepts,  technology, 
and  measurement  tools  used  in  the  corps  performance 
measurement  system.  The  training  will  include  lecture, 
class  discussion,  and  the  use  of  videotapes  to  provide 
actual  practice  in  using  the  observation  guides.  The 
videotapes  will  also  be  used  to  calibrate  the  raters  and  to 
collect  data  needed  to  evaluate  the  effectiveness  of  the 
training.  Some  observers  may  be  eliminated  if  they  do  not 
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successfully  complete  the  training  program  and  demonstrate 
acceptable  interrater  reliability  in  the  use  of  observation 
guides. 

Automatic  Battlefield  Event  Data  Collection  -Programs 

The  two  components  of  the  measurement  system  described 
previously  will  aid  in  collection  of  data  related  to  the 
synchronization  process  as  reflected  in  the  behavior  of 
commanders  and  their  staffs.  Additional  measurement  system 
components  will  be  developed  for  collecting  data  on 
synchronization  as  it  is  reflected  in  the  actions  of  units 
appearing  on  the  computerized  battlefield  provided  by  JESS. 
These  components  will  take  the  form  of  programs  to  be  added 
to  the  exercise  driver  (JESS)  which  automatically  record  and 
save  the  information  required  to  reconstruct  the  battlefield 
in  terms  of  disposition  and  status  of  friendly  and  OPFOR 
units  at  pre-determined  intervals  of  time.  This  will  allow 
a  reconstruction  of  "ground  truth"  for  different  phases  of 
the  operation.  While  not  providing  any  direct  measures  of 
synchronization,  the  data  produced  by  this  component  can  be 
analyzed  and/or  synthesized  to  derive  various  quantitative 
measures  related  to  rates  of  movement,  etc.  which  may  be 
used  to  calculate  or  infer  relative  degrees  of 
synchronization  of  operations.  Most  likely,  this  will 
involve  judgments  relating  unit  dispositions  and  status 
during  the  operation  to  expected  values  derived  from  the  OP 
Plan  and  commander's  concept  of  the  operation. 

PatU<?_9v.££gTne  Measures 

The  fourth  component  of  the  measurement  system  will  be 
battle  outcome  measures  produced  by  JESS.  Such  measures  may 
be  collected  and  saved  throughout  the  exercise  to  allow 
assessment  of  the  outcomes  of  key  engagements  which  occur  at 
different  points  in  time  in  addition  to  the  overall  end  of 
operation  outcome  measures. 

Ths-Prpfre ..  System 

The  four  components  of  the  measurement  system  described 
above,  represent  passive  components  of  a  non-obtrusive  data 
collection  system.  A  fifth  component  of  the  measurement 
system  is  the  probe  subsystem.  This  system  differs  from  the 
other  components  in  several  ways.  First,  it  must  be 
included  as  part  of  the  design  of  the  exercise  itself. 
Second,  it  has  a  direct  impact  on  the  conduct  of  the 
exercise  and  the  players  in  the  exercise.  Finally,  just  as 
JESS  is  the  driver  for  the  corps-level  exercise,  the  probe 
subsystem  is  the  "driver"  for  the  measurement  system.  The 
probe  system  will  be  constructed  to  generate  a  variety  of 
events/information  which  can  impact  directly  on  different 
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command  and  staff  elements  at  different  echelons.  The 
probes  will  be  entered  through  JESS  in  the  same  manner  as 
other  exercise  events,  often  in  the  form  of  specific  OPFOR 
actions.  The  probe  system  will  also  include  the  guidelines 
for  placement  of  observers  at  particular  information 
processing  nodes  as  well  as  procedures  for  alerting 
observers  that  the  probe  as  been  implemented.  The 
occurrence  of  the  probe  event  may  also  serve  as  a  trigger 
for  data  collection  programs  which  are  built  into  JESS  as 
part  of  the  automatic  data  collection  system. 

While  the  actual  performance  measurement  data  and  data 
collection  procedures  will  be  treated  in  more  detail  in 
later  sections,  it  is  important  to  examine  the  probe 
subsystem  in  some  detail  at  this  point.  The  actual  nature 
of  any  probe  event  or  sequence  of  events  will  be  tailored 
specifically  to  each  REDCOM  exercise.  The  insertion  of  such 
events  is  routinely  included  as  part  of  the  exercise.  The 
difference  between  the  use  of  such  probe  events  in  the  past 
and  those  included  in  the  current  measurement  system  is  that 
detailed  data  collection  efforts  will  be  cued  to  the 
implementation  of  the  probe  in  the  current  effort.  Figure 
7-2  provides  a  schematic  of  the  manner  in  which  the  probe 
subsystem  will  operate. 

As  noted  earlier,  a  probe  may  consist  of  a  single  event 
such  as  the  movement  of  a  particular  OPFOR  element  or 
implementation  of  an  unexpected  OPFOR  event  such  as  the  use 
of  chemical  weapons  or  a  sudden  air  strike.  Alternatively, 
a  probe  may  be  represented  by  a  sequence  of  events  or 
sequence  of  INTEL  messages  which,  if  synthesized  and 
interpreted  correctly,  provide  critical  information 
regarding  the  corps  operation.  The  probes  will  be 
constructed  and  entered  into  the  system  in  such  a  manner 
that  certain  staff  elements  should  be  stimulated. 

Given  the  focus  on  synchronization,  those  staff 
elements  at  corps  level  involved  in  battlefield 
synchronization  will  be  the  most  likely  groups  to  be 
targeted  for  stimulation  by  probe  events.  According  to  FC 
100-15,  Corps  Operations  the  staff  elements  most  directly 
involved  in  synchronization  of  the  corps  battle  include: 

■  The  corps  commander  or  his  designated 
representative 

■  The  G-3  staff 

■  The  G-2  staff 

■  The  Fire  Support  Element  representative 
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FIGURE  7-2.  REDCOM  Probe  System  Process 


■  The  tactical  air  control  party  staff 

■  The  electronic  warfare  section 

■  Other  service  representatives 

If  probes  are  inserted  to  examine  the  ability  of  the 
corps  to  maintain  synchronization  of  activities  in  response 
to  a  changing  battlefield,  observers  will  most  likely  be 
assigned  to  examine  the  response  of  the  staff  elements 
listed  above.  Observers  at  these  staff  elements  would  be 
alerted  when  the  probe  event  occurs  and  would  have 
observation  guides  with  prompts  identifying  specific  target 
behaviors  for  observation.  The  observations  would  be 
recorded  on  the  electronic  clipboard  and  automatically  time- 
tagged.  During  this  observation  period,  data  would  be 
automatically  extracted  from  JESS  to  record  battlefield 
events  and  relevant  messages  and  orders  entered  by 
controllers  at  the  JESS  workstations.  The  next  section  of 
the  report  provides  much  greater  detail  regarding  the  actual 
data  which  might  be  collected  during  REDCOM  exercises  and 
the  manner  in  which  the  data  will  be  transformed  into 
measures  of  performance  for  feedback  to  the  corps  commander 
and  his  staff. 
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SECTION  8 

DATA  COLLECTION,  ANALYSIS  AND  SYNTHESIS  FOR  THE  CORPS 
PERFORMANCE  MEASUREMENT  SYSTEM 


Overview 

The  exact  content  of  the  battlefield  event  timelines, 
observation  guides,  probes,  and  other  measurement  tools  used 
in  the  corps  performance  measurement  system  will  be  tailored 
to  each  REDCOM  exercise.  While  this  means  that  it  is 
impossible  to  present  specific  measurement  instruments  with 
the  exact  items  to  be  used,  it  is  possible  to  provide 
considerable  detail  on  the  types  of  data  and  data  collection 
procedures  used  in  the  performance  measurement  system.  In 
addition,  it  is  possible  to  discuss  the  manner  in  which  the 
data  will  be  synthesized  and  analyzed  to  derive  the  measures 
of  performance  which  provide  the  basis  for  the  feedback 
system.  This  section  of  the  report  will  provide  as  much 
detail  as  possible  regarding  the  data  collection  procedures 
and  data  analysis  procedures  required  to  derive  the  measures 
of  performance  called  for  by  the  measurement  concept 
presented  in  Section  7. 

The  corps  performance  measurement  system  will  involve 
collection  of  six  types  of  data.  The  data  to  be  collected 
include: 


■  Information  on  the  structure,  SO Ps  for  staff 
actions  and  other  relevant  data  on  the  corps 
participating  in  the  REDCOM  exercise 

■  Overall  combat  outcome  measures  generated  by  JESS 

■  Selected  engagement  outcome  measures  generated  by 
JESS 

■  Battlefield  event  and  action  data  extracted  from 
JESS 

■  Staff  actions,  products,  messages,  reports,  etc. 
extracted  from  JESS  and  collected/ recorded  by 
observers 

■  Answers  to  objective  questions  collected  either 
verbally  or  on  surveys  from  players  during  and/or 
after  the  exercise. 

Each  of  these  types  of  data  is  described  in  the 
paragraphs  below.  Following  the  description  of  each  of  the 
individual  measures  is  a  discussion  of  the  manner  in  which 
the  data  will  be  analyzed  and  synthesized  to  derive  measures 
of  corps  performance. 
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Corps  Structure  and  SOPs 


As  noted  earlier.  Army  corps  are  structured  to  meet  the 
needs  of  particular  theater  missions  and  contingency  plans. 
Furthermore,  the  SOPs  used  by  the  staff  to  accomplish  a 
variety  of  actions  related  to  the  processing  cf  critical 
information  may  vary  widely.  To  aid  in  the  development  of 
observation  guides  and  placement  of  observers  during  the 
exercise,  it  will  be  important  to  collect  data  on  the 
structure  of  the  corps  staff  and  SOPs  used  by  the  staff. 
This  data  will  be  collected  during  a  home-station  visit 
prior  to  the  start  of  the  RE DCOM  exercise. 

Combat  Outcome  Measures 

As  noted  earlier,  extreme  caution  must  be  observed  in 
using  combat  outcome  measures  produced  by  a  model  such  as 
JESS.  Nonetheless,  such  measures  must  be  incorporated  in 
the  corps  performance  measurement  system.  The  majority  of 
the  data  required  to  examine  combat  outcomes  can  be  taken 
directly  from  summary  reports  generated  by  the  JESS.  The 
basic  data  which  will  be  extracted  from  JESS  to  examine 
combat  outcomes  include: 

1.  Location  and  status  of  all  units  (BLUE  and  RED)  at 
the  start  of  the  exercise 

2.  The  final  location  and  status  of  all  units  (BLUE 
and  RED)  at  the  end  of  the  exercise 

3.  Total  personnel  casualty  figures  for  BLUE  and  RED 
forces  by  the  smallest  unit  of  analysis  possible 
(probably  Bn) 

4.  Total  BDA  figures  for  all  weapon  systems  (air  and 
ground)  of  both  RED  and  BLUE  forces.  If  possible, 
this  will  include  information  about  the  type  of 
system  making  each  kill. 

5.  All  logistics  data  on  ammunition,  fuel,  food,  etc. 
used  by  the  Corps  during  the  operation. 

6.  Data  on  all  supplies,  etc.  which  were  partially  or 
totally  distributed  but  not  consumed,  never 
distributed,  etc. 

7.  All  available  data  on  RED  casualties  inflicted  by 
each  BLUE  unit  (by  smallest  unit  possible) . 

8.  Start  and  end  times  for  the  exercise 

9.  Summary  data  on  fire  support  missions 
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10.  Summary  data  on  all  air  missions 
Selected  Engagement  Outcome  Data 

During  periods  of  data  collection  following  the 
implementation  of  probes  and  at  predetermined  periods  of 
focused  observation  suggested  by  the  planned  event 
timelines,  outcome  data  from  selected  engagements  and/or 
battles  will  be  collected  from  JESS.  The  data  to  be 
collected  for  these  engagements • includes : 

1.  Identification  and  location  of  all  units  involved 
in  the  battle  or  engagement 

2.  Start  time  for  the  engagement 

3.  Complete  unit  status  of  all  units  at  the  start  of 
the  engagement 

4.  Complete  records  of  all  casualties,  BDA,  etc. 
produced  during  the  engagement  for  all  units 
involved 

5.  Ending  time  for  the  engagement 

6.  Final  unit  status  for  all  units  involved  in  the 
engagement 

7.  Final  locations  of  all  units  involved  in  the 
engagement  immediately  following  the  end  of  the 
engagement 

8.  Summary  data  on  all  air  and  fire  support  missions 
called  for  during  the  engagement  by  both  RED  and 
BLUE  forces 

Battlefield  Event  andJtetien  Pat,, a 

Following  the  implementation  of  probes  or  based  on 
predetermined  periods  of  focused  observation,  selected 
battlefield  event  data  will  be  extracted  from  JESS.  The 
target  events  will  be  selected  based  on  the  BLUE  and  RED 
event  timelines  and  anticipated  responses  to  probe  events. 
The  exact  means  by  which  the  data  will  be  extracted  cannot 
be  specified  at  this  time.  One  option  is  to  alter  the  JESS 
program  to  dump  all  graphics  and  message  data  onto  tape  for 
later  playback.  A  second  option  is  to  clearly  specify 
target  events  or  actions  and  place  an  observer  at  the  master 
control  workstation  to  observe  all  graphics  and  message 
traffic.  When  the  event  occurs  the  observer  could  make  all 
necessary  records  manually.  Assuming  that  an  automated 
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recording  system  can  be  developed  for  the  JESS,  the 
following  types  of  data  will  be  recorded  (depending  on  the 
nature  of  the  target  event  or  action) : 

1.  Snapshot  of  the  entire  graphics  (locations, 
movement,  of  all  units  at  the  time  of  the  event) 

2.  Magnified  snapshot  of  the  area  in  which  the  event 
occurs 

3.  Complete  location  and  unit  status  data  of  all 
elements  involved  in  the  event 

4.  Starting  and  ending  time  of  the  target  event 

5.  Any  relevant  outcome  data  produced  by  the  event 
(See  list  of  selected  engagement  outcome  data 
above) 

Staff  Actions.  Reports.  Messages,  etc. 

At  selected  times  after  implementation  of  probes  or 
based  on  the  staff  event  timelines  developed  from  analysis 
of  the  corps  operations  plans,  observers  will  make  detailed 
recordings  of  staff  information  processing  actions.  In 
addition,  the  observers  may  obtain  copies  of  selected 
messages,  reports,  etc.  produced  by  staff  members.  All 
operations  plans,  written  orders,  etc.  will  be  collected  on 
a  routine  basis.  The  data  collected  by  observers  will  take 
three  different  forms. 

The  first  type  of  data  is  observations  related  to  staff 
information  processing  behaviors.  These  observations  will 
be  recorded  using  the  electronic  clipboards  described 
earlier.  While  the  exact  nature  of  the  data  collected  will 
be  determined  by  the  nature  of  the  corps  operation,  it  will 
most  likely  include  the  following  generic  elements: 

1.  Time  of  the  observation 

2.  Type  of  information  processing  action  observed 
(gathering  information,  relay,  transformation, 
analysis,  synthesis,  etc.) 

3.  Staff  element  under  observation 

4.  Essential  characteristics  of  information  being 
processed: 


■  Type  of  information 

■  Source 
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■  Media 

■  Etc. 


5.  Outcome  of  the  staff  action 

6.  Perceived  objective  of  the  action 

7.  Use  of  SOPs 

8.  Time  information  or  information  product  was  passed 
to  other  staff  elements 

9.  Identification  of  staff  elements  where  the 
information  or  products  were  sent 

10.  Media  or  means  by  which  the  information  was  sent 
to  other  staff  elements  (radio,  phone,  written 
message,  etc.) 

The  second  type  of  information  gathered  by  observers 
will  be  copies  of  messages,  reports,  plans,  orders,  etc. 
which  are  clearly  related  to  target  events  or  target  staff 
actions.  While  the  reports  will  not  be  physically  collected 
at  the  time  the  observer  notes  their  existence,  records  of 
their  existence  must  be  noted  and  arrangements  should  be 
made  with  the  exercise  controllers  to  obtain  copies  as  soon 
as  possible.  An  essential  element  of  data  which  observers 
must  collect  will  be  photographs  or  detailed  notes  on  the 
situation  maps  of  various  corps  staff  elements  (particularly 
the  G-3  and  G-2)  during  critical  points  or  phases  of  the 
operation.  These  maps  will  be  compared  to  "snapshots  of 
ground  truth"  taken  automatically  from  the  JESS. 

A  third  type  of  data  to  be  collected  by  observers 
during  focused  periods  of  observations  will  be  hard  copies 
of  controller  input  and  reports  produced  by  JESS.  These 
printouts  will  be  collected  from  selected  workstations 
representing  the  staff  and/or  maneuver  elements  on  which 
observations  are  centered.  Since  each  workstation  has  a 
printer,  this  will  produce  no  particular  burden  on  the 
controller.  Furthermore,  each  entry  and  report  is 
automatically  time  tagged  by  the  JESS.  Examples  of  the  type 
of  data  which  will  be  collected  using  this  method  include: 

1.  Controller  commands  to  move  units 

2.  Controller  requests  for  unit/logistics  status 
based  on  requests  from  players 

3.  Controller  commands  to  change  task  organization 

4.  Controller  requests  for  supplies,  etc. 
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5.  Controller  requests  for  fire  support  missions 

6.  Breaching  of  minefield  commands 

7.  Controller  requests  for  air  support 

8.  Requests  to  draw  supplies 

9.  Crossleveling  of  supplies  requests 

10.  Request  to  create  supply  convoy 

11.  Orders  to  change  maintenance  support  unit 
Plaver  Answers  to  Objective  Questions 

The  final  type  of  data  to  be  collected  during  the 
REDCOM  exercise  will  be  data  collected  from  key  players  on 
the  corps  staff  and  division  commanders  and  staff  members. 
The  information  will  be  in  the  form  of  answers  to  a  small 
number  of  objective  questions  designed  to  assess  their 
understanding  of  the  corps  commander's  concept  of  the 

operation  and  his  intent.  While  these  questions  will 
obviously  be  tailored  to  each  exercise,  the  general  purpose 
of  the  questions  will  be  to  determine  the  extent  to  which 
each  key  player  understands: 

1.  The  overall  intent  or  objective  of  the  corps 
mission  (as  stated  by  the  corps  commander) 

2.  The  role  of  his  organizational  element  in  the 

corps  operation 

3.  The  timing  or  importance  of  key  events  or  actions 
in  which  his  organizational  element  is  involved 

4.  The  objective  or  intent  of  specific  key  events 

5.  Any  specific  requirements  for  his  organizational 

element  with  regard  to  location  at  a  particular 
point  in  time 

6.  Requirements  for  coordination/synchronization  of 
his  actions  with  those  of  another  element  in  the 
operation 

7.  Requirements  or  importance  of  sharing  specific 
types  of  information  with  other  individuals  or 
organizational  elements. 
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The  data  concerning  understanding  of  the  commander's 
intent  and  concept  of  the  operations  will  be  collected  from 
players  in  such  a  way  as  to  avoid  any  interference  in  the 
actual  conduct  of  the  exercise.  This  may  mean  that  data 
cannot  be  collected  from  certain  key  players  until 
completion  of  the  exercise.  While  this  may  create  some 
problems  because  the  players  have  developed  very  keen 
hindsight,  it  is  a  problem  which  is  unlikely  to  be  easily 
solved,  particularly  if  data  is  to  be  collected  from 
division  commanders  and/or  division  and  corps  principal 
staff  officers  (6-3  and  G-2). 

Data  Integration  and  Derivation  of  Measures  of  Performance 

The  data  described  above  represents  the  raw  material 
used  in  the  derivation  of  the  measures  of  corps  performance 
which  will  be  used  in  providing  feedback  to  the  corps 
commander  and  his  staff.  This  section  of  the  report  will 
provide  as  much  detail  as  possible  on  the  manner  in  which 
the  raw  data  will  be  analyzed  and  synthesized  to  derive  the 
performance  measures.  The  discussion  will  begin  with  the 
criterion  measures  which  will  be  used  to  provide  feedback  as 
to  "how  well"  the  corps  performed. 

Criterion  Measures 

Most  of  the  criterion  measures  will  be  obtained 
directly  by  summarizing  or  combining  combat  outcome  data 
arithmetically.  Not  all  measures  will  be  presented  for  each 
corps  operation.  The  importance  of  a  particular  outcome 
measure  is  dependent  on  the  nature  of  the  mission  which  the 
corps  is  performing.  The  outcome  measures  will  be  organized 
around  the  structure  provided  by  the  attributes  involved  in 
a  MET-T  type  of  mission  analysis.  This  mission  outcome 
structure  was  developed  as  part  of  the  NTC  performance 
measurement  system  and  has  met  with  considerable  support  as 
a  meaningful  way  to  analyze  outcome  data.  Figure  8-1 
provides  the  general  structure  for  synthesizing  and 
presenting  the  outcome  data.  The  first  two  columns  of  the 
outcome  data  matrix  will  be  completed  based  upon  an  analysis 
of  the  corps  operations  plan  and  discussions  with  the  corps 
commander  as  to  what  he  considers  to  be  acceptable  standards 
for  successful  completion  of  the  mission  as  measured  by  the 
attributes  of  terrain,  resources,  and  time.  The  criterion 
measures  used  to  complete  the  matrix  and  the  manner  in  which 
they  will  be  calculated  from  the  raw  data  are  listed  below. 
Unless  stated  otherwise,  the  outcome  measures  will  be  taken 
directly  from  summary  reports  generated  by  the  JESS  and  will 
require  relatively  little  transformation.  The  data  used  to 
complete  the  data  synthesis  matrix  illustrated  in  Figure  8-1 
will  include: 
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FIGURE  «-l.  Structure  For  Synthesizing  Outcome  Measures 


Calculated  from  Original  Positions  and  Final  Positions  of 
BLUEFOR  AND  OPFOR  units 


RESOURCES : 

Friendly  fBLUE)  Resources  Data 

■  Blue  Force  Casualties: 

-  By  Battalion 

-  By  Bde 

-  By  Division 

-  Total 


■  Blue  Force  BDA: 

-  By  Unit  and  System 

-  Total  By  System 

■  Total  BLUE  FORCE  Ammunition  expended: 

-  By  unit  by  Type 

-  Total  by  type 

■  Fuel  Expended: 

-  By  unit 

-  Total 

Enemy,  IQPEQE)  Resource  P?t3 

■  OPFOR  Casualties: 

-  By  Unit 

-  Total 

■  OPFOR  BDA: 

-  By  unit  and  system 

-  Total  by  system 


■  Total  BIAJE  FORCE  Ammunition  expended: 

-  By  unit  by  Type 

-  Total  by  type 

■  Fuel  Expended: 

-  By  unit 

-  Total 
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TIME  FACTORS 


Time  to  Reach  Objective  (BLUEFOR _ PFJEESIYE _ HISSIQN)  . 

Calculated  from  the  time  the  exercise  begins  to  the  time  the 
forward  maneuver  elements  of  the  corps  has  occupied  the  hex 
at  the  center  of  its  objective  (or  end  of  exercise) . 

Rate  of  Movement  jfor  the  Corps  (BLUEFOR  OFFENSIVE -MISSION) . 
Calculated  by  determining  the  average  distance  traveled  by 
forward  maneuver  elements  from  their  starting  to  ending 
locations  and  then  dividing  this  distance  by  the  time  taken 
to  reach  the  objective  (above) . 

Time  Required  for  the  OPFOR  to  Reach  Objective _ (ELUEEfiB 

DEFENSIVE  MISSION! .  Calculated  from  the  time  the  exercise 
begins  to  the  time  the  forward  maneuver  elements  of  the 
OPFOR  have  occupied  the  hex  in  the  center  of  their 
objective. 

Mission  Accomplishment 

The  corps  commander  or  the  director  of  the  REDCOM 
exercise  may  wish  to  make  a  general  statement  concerning  the 
extent  to  which  the  corps  has  accomplished  its  mission. 
Such  a  statement  may  be  a  simple  yes  or  no  type  of  answer 
for  a  relatively  straightforward  mission  or  a  fairly  complex 
answer  which  requires  certain  qualifying  statements  for  a 
complex  mission.  In  any  case,  the  judgment  will  probably  be 
based  on  an  assessment  of  all  of  the  criterion  measures 
described  above  plus  the  corps  commander's  assessment  of  the 
standards  which  define  successful  performance  on  each  of  the 
criteria. 

P?£f pjE5>an<?e -Measures  Enfl,  synchronization  indis?? 

The  second  group  of  performance  measures  are  those 
which  provide  an  objective  "picture"  of  the  unfolding  of  the 
operation  and  the  degree  to  which  the  operation  was  executed 
as  planned.  These  measures  are  derived  indirectly  from  the 
raw  data.  Figure  8-2  provides  a  schematic  of  the  process 
used  to  derive  the  performance  measures.  The  operations 
plans  are  first  analyzed  to  construct  an  event  timeline  with 
synchronization  relationships. 

Based  on  the  battlefield  events  included  in  the 
timeline,  a  planned  staff  action  timeline  is  derived  based 
on  inferences  as  to  staff  actions  and  timing  of  staff 
actions  required  at  corps  level  to  execute  the  battlefield 
events  in  the  operations  plan.  While  construction  of  the 
planned  battlefield  event  timelines  is  a  fairly  objective 
task,  the  construction  of  the  latter  predicted  staff  action 
timeline  requires  extensive  knowledge  of  corps  staff 


8-10 


FIGURE  8-2.  Derivation  Of  Performance  Measures 


operations  as  well  as  knowledge  concerning  the  SOPs  for 
staff  operations  in  the  corps  being  assessed. 

Accompanying  the  battlefield  event  timeline  should  be  a 
complete  copy  of  the  corps  control  graphics  with  symbology 
and  information  of  projected  unit  locations  at  critical 
times  during  the  operation.  These  three  tools  represent  the 
baseline  data  from  which  calculations  will  be  made  to  assess 
synchronization  of  the  corps  operations. 

As  noted  previously,  a  parallel  OPFOR  battlefield  event 
timeline  will  also  be  constructed.  Accompanying  this 
timeline  will  be  a  map  with  all  relevant  OPFOR  graphics. 

After  the  exercise  has  been  completed,  the  battlefield 
event  and  action  data  extracted  from  JESS  and  the  staff 
action  data  collected  by  observers  using  electronic 
clipboards  will  be  used  to  construct  a  second  set  of  event 
timelines.  The  second  set  of  event  timelines  will  consist 
of  actual  battlefield  events  and  actual  staff  actions 
related  to  the  corps  operation.  The  photographs,  observer 
notes,  and  unit  locations  reports  extracted  from  JESS  at 
particular  stages  in  the  exercise  will  also  be  synthesized 
to  form  a  set  of  "ground  truth"  situation  maps. 

Simply  presenting  the  actual  battlefield  event  and 
staff  action  timelines  and  "ground  truth"  situation  maps 
will  have  significant  feedback  value  for  the  corps.  In 
addition  to  such  qualitative  summaries  of  raw  data,  the 
following  quantitative  measures  of  performance  will  be 
calculated: 

■  Percentage  of  planned  events  successfully  executed 

during  the  elapsed  time  of  the  exercise.  This 
will  be  calculated  by  dividing  the  total  number  of 
discrete  events  identified  on  the  baseline 
battlefield  event  timeline  by  the  number  of  these 
events  which  are  successfully  accomplished  as 
indicated  by  the  actual  battlefield  event 

timeline.  The  definition  of  successful  completion 
will  be  based  upon  the  corps  commanders'  prior 
statements  concerning  mission  standards  and  the 
selected  engagement  outcome  data  extracted  from 
JESS. 

■  Time  deviations  for  planned  and  actual  events  will 
be  calculated  based  on  the  baseline  and  actual 
event  time  lines 

■  Number  of  unplanned  key  events.  This  measure  of 
performance  may  take  several  forms.  It  will  be 
derived  based  on  observations  and  data  extracts 
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from  JESS  made  during  observation  periods 
following  probes  and  other  predetermined 
observation  periods.  The  events  will  be  found  on 
the  actual  event  timeline  and  may  include  probe 
events,  unexpected  BLUEFOR  actions  (positive  or 
negative),  and  unanticipated  OPFCR  actions.  This 
data  will  be  presented  to  stimulate  discussion  of 
contingency  planning,  actions  demonstrating 
initiative  and  understanding  of  the  commander's 
concept,  etc.  The  OPFOR  battlefield  event 
timelines  and  graphics  will  be  particularly 
important  for  this  feedback. 

■  Deviation  in  planned  and  actual  unit  locations 
will  be  calculated  based  on  the  baseline  map  and 
actual  situation  maps.  A  limited  amount  of  this 
data  will  be  calculated  and  presented.  The  data 
will  be  presented  only  for  instances  in  which  unit 
location  appeared  to  be  a  major  factor  affecting 
synchronization  (or  lack  thereof  of  the 
operation) . 

■  Qualitative  comparisons  or  simple  presentation  of 
photographs  or  copies  of  the  G-2  and  G-3  sitmaps 
and  snapshots  of  "ground  truth"  of  the  situation 
extracted  at  key  points  in  time  from  JESS  will  be 
used  to  demonstrate  the  accuracy  of  the 
intelligence  process  and  corps  processing  of 
information. 

SynchronisatiQii  inflice? 

Examination  of  the  battlefield  and  staff  event 
timelines  and  the  corps  operations  graphics  and  "ground 
truth"  sit  maps  should  allow  a  qualitative  assessment  of  the 
degree  of  synchronization  required  and  achieved  in  an 
operation.  The  authors  recommend  that  the  timelines  should 
be  used  as  the  primary  source  of  feedback  on  synchronization 
to  the  corps. 

On  the  other  hand,  the  authors  also  recognize  that  some 
quantification  of  synchronization  may  be  desired  for 
research  purposes.  Two  potential  indices  may  be  derived 
from  the  use  of  data  collected  in  the  proposed  performance 
measurement  system.  The  first  index  represents  a  measure 
of  the  degree  of  synchronization  required  in  the  plan  for 
the  operation  while  the  second  represents  an  indication  of 
the  degree  of  synchronization  achieved.  The  reader  should 
be  cautioned  that  the  methods  for  calculating  these  indices 
have  yet  to  be  tested  and  are  related  to  the  operational 
definition  of  the  synchronization  concept  discussed  earlier 
in  the  report. 
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SynghrgnigaUpn-Bequired  in  Plan 


The  calculation  of  this  index  requires  the  use  of  the 
BLUE  battlefield  event  timeline,  the  operations  graphics, 
and  a  copy  of  the  concept  of  the  operation.  The  index  is 
calculated  using  the  following  equation: 

Req.  Synch.  ■  (Percent  of  Cross  Functional  events  +  Percent 
of  coordinated  location  events)  x  Time 
criticality  factor 


Where: 

The  percent  of  cross  functional  events  is  equal  to  the 
total  number  of  events  on  the  planned  battlefield  event 
timeline  that  are  linked  across  functional  areas  divided  by 
the  total  number  of  events  on  the  timeline. 

The  percent  of  coordinated  location  events  is  equal  to 
the  total  number  of  events  on  the  planned  battlefield  event 
timeline  that  the  corps  graphics  and  operations  plan  call 
for  supporting  actions  by  at  least  two  battalion  size  units 
or  elements  from  two  different  battalion  size  units  which 
must  be  in  particular  locations  divided  by  the  total  number 
of  events  on  the  same  timeline. 

The  time  criticality  factor  is  equal  to  the  sum  of  the 
time  criticality  values  for  all  cross-functional  events 
divided  by  the  total  number  of  cross-functional  events.  The 
time  criticality  value  for  a  cross  functional  event  is  a  an 
indicator  of  the  extent  to  which  the  actions  of  the  cross¬ 
functional  elements  must  be  synchronized  in  time.  The 
values  are  assigned  using  the  information  in  Table  8-1. 
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Table  8-1.  Time  Criticality  Values 


Degree  of 

Time  Synchronization  Time 

n  Plan  Criticality  Factors 


Window  for  synchronization  is  one 

hour  or  more  1 

Window  for  synchronization  is  between 

30  minutes  and  59  minutes  2 

Window  for  synchronization  is  between 

15  and  29  minutes  3 

Window  for  synchronization  is  between 

7.5  and  14  minutes  4 

Window  for  synchronization  fs  less  than 
7 . 5  minutes  5 
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Synchronization  Achieved  in  the  Operation 


This  index  measures  the  degree  of  planned 
synchronization  in  the  operation  which  was  actually 
accomplished  in  the  execution  of  the  operation.  This  index 
is  calculated  using  the  following  equation: 

Synch  Achieved  ■  (Percent  of  actual  cross-functional  events 

+  Percent  of  actual  coordinated  location 
events)  x  Actual  time  synchronization 
factor 


Where: 

The  percent  of  actual  cross-functional  events  is  equal 
to  the  total  number  of  events  in  which  the  planned  cross¬ 
functional  actions  occur  divided  by  the  total  number  of 
planned  cross-functional  events. 

The  percent  of  actual  coordinated  location  events  is 
equal  to  the  number  of  planned  coordinated  action  events 
when  all  units  are  at  planned  locations  divided  by  the  total 
number  of  such  events  that  were  planned. 

The  time  synchronization  factor  is  equal  to  the  product 
of  the  percentage  of  planned  events  requiring  time 
synchronization  which  are  executed  within  the  allotted  time 
window  multiplied  by  the  average  time  criticality  factor  of 
the  successfully  executed  events. 

Once  again,  it  must  be  stressed  that  these  are  only 
tentative  indices  of  synchronization  which  are  to  be  used 
for  research  purposes,  not  for  feedback  to  the  corps.  The 
indices  are  simply  a  means  of  summarizing  data  related  to 
the  elements  of  synchronization  (concentration  of  effects  in 
time  and  space)  identified  in  the  review  of  doctrine.  If 
research  indicates  that  the  indices  provide  a  valid  measure 
of  synchronization  then  they  may  later  be  used  as  a  summary 
statistic  provided  as  feedback. 

Diagnostic  Process  Measures 

The  third  group  of  performance  measures  are  those 
measures  which  aid  in  explaining  why  operations  did  not  go 
as  planned.  The  bulk  of  these  measures  focus  on  the  data 
from  observations  and  automatic  data  collection  on 
information  processing  variables  and  the  questionnaires 
examining  the  understanding  of  the  commander's  intent  and 
concept  of  the  operation. 
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The  first  group  of  these  diagnostic  measures  includes 
data  related  to  the  degree  to  which  subordinate  commanders 
and  the  corps  staff  understood  the  corps  commander's  intent 
and  concept  of  the  operation.  The  measures  will  be 
calculated  from  scores  on  the  objective  questions  answered 
by  players  and  include: 

■  The  percentage  of  players  surveyed  with  accurate 
answers  to  each  question; 

« 

■  Some  measure  of  the  average  degree  of  discrepancy 
present  in  incorrect  answers  concerning  location, 
time,  or  other  quantifiable  attributes  of  the 
operation; 

■  Frequency  and  percentages  of  the  types  of 
discrepancies  found  in  the  players  answers. 

The  second  major  group  of  diagnostic  measures  are 
measures  of  information  processing  performance  collected  by 
observers  following  the  insertion  of  probes.  This 
information  will  include  two  types  of  data.  The  first  will 
be  a  flow  chart  similar  to  a  staff  action  micro-timeline 
which  traces  all  available  data  on  the  actions  of  the 
commanders  and  staff  stimulated  by  the  probe.  The  second 
type  of  data  will  be  average  values  for  the  performance  of 
the  staff  on  various  aspects  of  information  processing 
measured  by  observers.  This  information  will  be  presented 
by  each  major  staff  element  assuming  adequate  data  has  been 
collected  for  each  element  during  the  exercise.  The  data 
will  include: 

■  Frequency  and  percentages  of  various  types  of 
information  processed  by  each  staff  element; 

■  Frequency  of  occurrence  and  percentage  of  time 
spent  by  various  staff  elements  in  various  types 
of  information  processing  behaviors  (gathering, 
relaying,  synthesizing,  etc.) 

■  Average  times  required  to  complete  various  types 
of  information  processing  activities 

■  Average  time  required  for  communication  of 
information  to  and  from  different  staff  elements 

■  Percentage  of  observed  staff  actions  following 
corps  SOPs 

■  Cross  tabulations  of  various  information 
processing  measures  by  Actions  following  or  not 
following  SOPs. 
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Summary 


The  data  elements  and  performance  measures  described 
above  represent  a  first-cut  attempt  at  the  development  of 
operational  measures  of  corps  performance  related  to  the 
synchronization-based  model  of  corps  performance  presented 
in  Figure  4-3.  Table  8-2  presents  the  eight  performance 
factors  identified  in  the  corps  performance  model.  The 
measures  of  performance  described  above  will  provide 
feedback  to  the  corps  commander  and  his  staff  on  five  or 
possibly  six  of  these  performance  factors.  More 
specifically,  the  measures  of  performance  will  allow  the 
following  questions  to  be  addressed  concerning  corps 
performance: 

1.  The  extent  to  which  the  commander  has  communicated 

and  his  subordinates  have  understood  the 

commander's  intent  and  concept  of  the  operation. 

2.  The  degree  to  which  the  staff  coordinates  and 
synchronizes  their  actions  in  completing  the  corps 
operations  plan  and  during  the  execution  of  the 
plan. 

3.  The  extent  to  which  various  elements  of  the  corps 
execute  their  assigned  missions  as  measured  by  the 
selected  outcome  measures  extracted  from  JESS  and 
actual  battlefield  and  staff  event  matrices 
constructed  from  various  data. 

4.  The  ability  of  commanders  and  staff  to  adjust  to 
changes  as  measured  by  information  processing  and 
command  actions  following  the  implementation  of 
probe  events. 

5.  The  degree  to  which  commanders  and  their  staffs 
plan  and  coordinated  logistical  support  to 
maneuver  elements  as  measured  by  admin/log  reports 
extracted  from  JESS. 

Three  additional  performance  factors  contained  in 
Table  8-2  are  not  directly  addressed  by  the  proposed  corps 
performance  measurement  system.  The  nature  of  the  REDCOM 
exercise  environment  and  practical  considerations  regarding 
the  nature  of  feedback  likely  to  be  acceptable  to  players  in 
a  corps  exercise  preclude  the  development  of  measures  of 
performance  for  the  performance  factors  of  adequacy  of 
information  used  as  input  to  the  corps  plan,  soundness  of 
the  corps  operations  plan,  and  the  degree  to  which 
communications  are  maintained  during  the  operation. 
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The  nature  of  the  RE DCOM  exercise  does  not  allow  one  to 
examine  the  corps  intelligence  process  which  would  normally 
gather  information  used  as  input  in  the  development  of  the 
corps  plan  of  operations.  Currently,  this  information  will 
be  supplied  by  REDCOM.  In  the  future,  if  JTLS  is  used  for 
situation  development,  it  may  be  possible  to  evaluate  this 
performance  factor. 


Table  8*2.  Performance  Factors  Identified  in 
Corps  Performance  Model 


■  Accuracy  and  completeness  of  information  used  to 
develop  the  corps  operations  plan 

■  Soundness,  adequacy,  etc.  of  the  operations  plan 
developed  by  the  commander  and  the  G-3 

■  The  degree  to  which  the  plan  and  intent  of  the 
commander  is  clearly  communicated  and  understood 
by  subordinate  commanders  and  the  corps  staff 

■  The  degree  to  which  the  corps  staff  coordinates 
and  synchronizes  their  actions  in  preparing  for 
and  implementing  the  plan 

■  The  extent  to  which  subordinate  echelons 

(particularly  the  maneuver  elements)  execute  their 
assigned  missions 

■  The  degree  to  which  lateral  and  vertical 

communications  are  maintained  during  execution  of 
the  operation 

■  The  ability  of  the  commanders  and  staffs, 

particularly  at  upper  levels  (division  and  corps) 
to  adjust  plan  in  response  to  changes  on  the 
battlefield 

■  Logistical  support  to  maneuver  elements 


No  attempt  is  made  in  the  current  performance 
measurement  system  to  evaluate  the  soundness  or  adequacy  of 
the  corps  operations  plan.  While  the  planned  battlefield 
event  and  staff  action  matrices  provide  a  very  explicit 
diagram  of  the  plan,  the  authors  do  not  consider  it  feasible 
for  the  controllers  involved  in  the  REDCOM  exercise  to  pass 
judgment  on  the  adequacy  of  the  plan.  If  the  plan  is 
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executed  well  and  the  result  is  failure,  problems  with  the 
plan  are  likely  to  become  self-evident.  If  the  plan  is  not 
executed  well,  it  is  likely  to  be  difficult  to  pass  any 
judgment  on  the  plan  with  the  exception  of  an  assessment 
that  the  plan  was  too  complex  for  successful  execution.  In 
either  case,  such  judgments  should  be  made  by  the  corps 
commander  and  his  staff;  not  the  observers  and  controllers 
in  the  exercise. 

The  final  performance  factor  not  directly  addressed  by 
the  proposed  measurement  system  is  the  degree  to  which 
vertical  and  horizontal  communications  are  maintained  during 
the  execution  of  the  operation.  The  primary  focus  of  this 
factor  as  described  in  the  model  is  on  tactical 
communications  which  allow  lateral  coordination  among 
maneuver  elements  and  vertical  communication  of  spot  reports 
and  orders.  The  recording  of  such  communications  is 
difficult  and  not  realistic  in  the  Bold  Venture  exercises 
which  are  primarily  CPXs.  On  the  other  hand,  the  degree  to 
which  communications  are  maintained  in  the  FTX  portion  of 
the  Bold  Eagle  exercises  should  be  noted  by  observers  and 
assessed  at  least  qualitatively  during  actions  which 
required  lateral  coordination  of  maneuver  elements. 

While  an  initial  illustration  of  the  viability  of 
portions  of  the  measurement  concept  and  data  collection 
procedures  described  in  Sections  7  and  8  was  demonstrated 
through  the  Inchon  Landing  example,  the  measurement  concept 
and  data  collection  and  analysis  procedures  require 
extensive  validation.  The  final  section  of  the  report 
presents  a  plan  for  conducting  this  evaluation  and 
validation  of  the  measurement  concept  for  assessing  corps 
performance. 
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SECTION  9 

VALIDATION  APPROACH 


Introduction 

The  final  steo  in  the  develoDment  of  an  organizational 
performance  measurement  system  is  the  validation  of  the 
performance  measures  and  assessment  of  the  utility  of  the 
feedback  system.  Validation  of  any  performance  measurement 
system  is  typically  a  difficult  task.  The  basic  problem 
Involved  in  validation  of  a  performance  measurement  system  is 
that  the  system  beinq  validated  was  probably  developed 
because  there  were  no  acceptable  performance  measures  already 
in  existence.  Therefore#  in  most  cases#  it  is  not  practical 
to  use  a  criterion-related  validation  design  to  validate 
performance  criterion  measures — there  are  no  measureable 
criteria  to  validate  the  performance  measures  against. 

While  difficulties  exist#  there  are  a  number  of  ways  to 
evaluate  (if  not  validate)  a  performance  measurement  system. 
Typically#  the  evaluation  should  take  place  in  two  phases. 
The  first  phase  of  the  evaluation  represents  a  formative 
evaluation.  In  this  phase#  the  major  focus  is  to  collect 
information  regarding  problems  involved  in  the  actual 
implementation  of  the  performance  measurement  system.  The 
attempt  is  not  to  evaluate  the  quality  of  the  data  produced 
by  the  system.  Rather#  the  objective  is  to  examine  Droblems 
which  mav  arise  in  the  collection  of  data.  This  evaluation 
would  include  careful  observation  and  recordina  of  Droblems 
such  as  lack  of  clarity  in  observer  instructions#  hardware  or 
software  problems  in  automated  data  collection  programs# 

logistical  problems  in  providing  support  to  observers#  the 
need  for  more  observer  training#  evaluation  of  potential 
interference  in  organizational  performance  created  by  the 
measurement  process#  etc.  The  results  of  the  formative 
evaluation  can  be  used  to  revise  performance  measures# 
develop  new  measures  or  data  collection  procedures#  eliminate 
problem  measures#  identify  data  collector  training 

requirements#  or  perhaps  to  make  a  decision  on  the 
feasibility  of  implementing  any  performance  measurement 
process. 

The  second  phase  of  the  performance  measurement 
evaluation  process  is  the  summative  evaluation  phase.  During 
this  second  phase#  the  evaluation  should  focus  on  the  quality 
of  the  data  produced  by  the  system#  user  reactions  to  the 
system#  and  the  extent  to  which  the  performance  measurment 
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system  meets  the  oriainal  design  requirements  or  objectives. 
The  summative  evaluation  phase  includes  an  assessment  of  the 
"validity”  of  the  performance  measurement  system. 

Criteria  for  Criteria 

Before  addressing  the  specific  evaluation  design  for  the 
corps  performance  measurement  system*  it  is  important  to 
consider  the  range  of  potential  criteria  which  miaht  be  used 
to  evaluate  an  oraanizational  Derformance  measurement  svstem. 
As  noted  bv  Bernardin  and  Beattv  (1984)*  there  are  a  varietv 
of  criteria  which  can  be  used  to  assess  a  performance 
measurement  system.  In  selecting  the  criteria  used  in  the 
evaluation*  it  is  important  to  keep  in  mind  the  purpose  or 
objectives  of  the  performance  measurement  system.  The 
criteria  selected  for  evaluation  should  provide  data  related 
to  how  well  the  performance  measurement  svstem  meets  the 
objectives  for  which  it  was  develoc>ed. 

The  current  authors  were  able  to  identifv  at  least  six 
criteria  which  miaht  be  considered  in  the  evaluation  of  a 
perforamnce  measurement  system  such  as  the  corps  performance 
measurement  system  developed  in  the  current  research  effort. 
The  first  two  criteria  are  traditional  psychometric  criteria 
which  may  be  quantified  using  various  statistical  and 
psychometric  procedures.  The  four  remaining  criteria  vary  in 
terms  of  the  degree  to  which  they  may  be  quantified  and  the 
degree  to  which  they  represent  psychometric  vs. 
organizational  concerns  regarding  the  performance  measurement 
system. 

The  first  criteria  to  be  examined  with  regard  to  any 
measurement  procedure  or  instrument  regards  the  reliability 
of  the  measurement  process.  Reliability  refers  to  the 
stability  or  consistency  of  the  measurement  process. 
Traditionally*  reliability  is  measured  in  one  of  three  ways: 
test-retest  procedures  (stability  over  time)*  internal 
consistency  of  items  purported  to  measure  the  same  construct* 
or  interrater  reliability  (consistency  in  ratings  across 
different  observers  making  the  same  observations). 

The  dynamic  nature  of  the  phenomena  under  study  in  the 
current  effort  (corps  performance)  negates  the  possibility  of 
using  any  form  of  test-retest  reliability  measures  to  assess 
the  performance  measures.  The  exception  to  this  might  be  an 
assessment  of  the  reliabiltiy  or  variability  of  combat 
outcomes  produced  by  probablistic  algorithms  used  in  JESS. 
At  the  current  time*  the  authors  are  not  aware  of  data 
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documenting  the  variablity  produced  by  such  factors.  The 
limited  number  of  "instruments”  or  questionnaires  used  in  the 
corps  performance  measurement  system  limits  the  applicability 
of  internal  consistency  measures  of  reliability  also. 


'On  the  other  hand/  measures  of  the  degree  to  which  there 
is  interrater  agreement  in  the  observer  recording  procedures 
is  a  critical  factor  to  be  examined  in  the  corps  performance 
measurement  system.  As  noted  earlier/  observers  will  require 
trainina  to  use  the  corps  performance  measurement  system. 
Part  of  the  trainina  should  include  the  use  of  video  tapes  to 
calibrate  th  *  raters  and  assess  interrater  reliability. 
Multiple  observer s  should  be  used  during  the  validation 
period  to  examine  the  interrater  reliability  of  the  various 
observer  procedures  and  instruments  in  a  field  environment  in 
addition  to  the  data  collected  during  rater  training. 

The  second  traditional  psychometric  criterion  to  be  used 
in  evaluating  a  measurement  procedure  is  to  examine  the 
validity  of  the  system.  As  noted  above/  problems  arise  in 
using  the  standard  criterion  related  validity  strategy.  If 
independent  measures  of  corps  performance  exist  which  are 
considered  reliable  and  valid/  the  measures  produced  in  the 
current  system  could  be  correlated  with  these  other  criterion 
measures.  The  authors  recoanize  that  such  measures  do  not 
currently  exist  and  are  not  likely  to  exist  unless  the  United 
States  Army  becomes  engaged  in  an  actual  armed  conflict. 
Other  validition  strategies  offer  more  opportunities. 
Perhaps  the  most  likely  validity  which  might  be  demonstrated 
would  be  content  validity.  Content  validity  refers  to  the 
extent  to  which  the  actual  performance  domain  of  the 
organization  has  been  sampled  adequately  by  the  measurement 
procedures  used  in  the  performance  measurement  system.  When 
establishing  content  validity  of  an  individual  performance 
measurement  system/  the  evidence  used  to  demonstrate  content 
validity  is  a  thorough  job  task  analysis.  The  doctrine 
review  and  use  of  subject  matter  experts  in  development  of 
the  current  corps  performance  measurment  system  provide  some 
evidence  of  content  validity.  More  systematic  and 
comprehensive  collection  of  data  on  critical  tasks  and 
functions  performed  at  the  corps  level  would  provide 
additional  evidence  regarding  content  validity. 
Unfortunately/  there  is  no  single  statistic  such  as  a 
correlation  coefficient  which  represents  a  measure  of  content 
validity.  The  degree  to  which  such  validity  has  been 
established  requires  a  subjective  judgment  based  on  all 
available  data  described  above. 
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A  final  option  with  regard  to  establishing  validity  of 
the  performance  measures  is  to  use  a  construct  validity 
approach  as  represented  by  demonstration  of  convergent  and 
divergent  validity.  This  approach  requires  that  the 
researcher  use  multiple  methods  and  measures  to  develop  a 
network  of  correlations  which  should  contain  both  significant 
and  nonsignficant  relationships  with  the  performance  measures 
being  evaluated.  If  the  performance  measures  are  "construct 
valid"  they  should  have  higher  correlations  with  other 
measures  of  similar  performance  constructs  than  with  similar 
measurement  procedures  which  purportedly  measure  different 
constructs. 

Another  criteria  which  may  be  used  in  the  evaluation  of 
a  performance  measurement  system  is  the  degree  to  which  the 
organizations  being  assessed  show  variability  on  the 
performance  measures.  An  assumption  made  in  the  use  of  this 
criteria  is  that  performance  measurement  systems  which  allow 
one  to  discriminate  levels  of  performance  amona  different 
organizations  are  more  effective  measurement  systems.  This 
criteria  is  particularly  important  when  the  performance 
measurement  system  has  been  developed  for  evaluative  purposes 
such  as  assessment  of  unit  readiness. 

A  fourth  criteria  to  be  considered  in  evaluating  the 
performance  measurement  system  is  the  user  acceptance  and 
user  reaction  to  the  system.  This  criteria  includes  an 
assessment  of  user  reactions  and  evaluations  of  the  utility 
of  feedback  provided  by  the  performance  measurement  system. 
To  a  certain  extent/  the  user  reactions  are  likely  to  be 
related  to  the  degree  of  face  validity  of  the  performance 
measures  and  the  degree  of  perceived  relevance  of  the 
feedback  to  the  organizational  members'  personal  and 
organizational  goals.  Criterian  is  particularly  important 
when  the  performance  measurement  system  has  been  developed 
for  training  purposes. 

A  fifth  criteria  which  must  be  considered  in  an 
organizational  performance  measurement  system  such  as  the 
corps  performance  measurement  system  is  the  cost  and  ease  of 
use  of  the  system.  The  cost  must  be  considered  in  terms  of 
capital  outlay#  personnel  costs#  time#  etc.  The  personnel 
costs  must  be  examined  carefully  in  terms  of  the  types  of 
qualifications  required  for  observers  as  well  as  the  number 
of  data  collectors  required.  Often#  the  practicality  of  a 
system  is  limited  if  it  requires  sophisticated  skills  for 
observers. 
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The  final  criteria  which  should  be  assessed  in  the 
evaluation  of  a  performance  measurement  system  desianed  for 
trainina  purposes  is  whether  feedback  from  the  system  has  a 
positive  effect  on  subsequent  performance.  This  may  be 
difficult  to  assess  in  instances  such  as  the  REDCOM  exercise 
environment  in  which  feedback  will  not  be  provided  until 
after  the  exercise  is  complete. 

Evaluation  Strategy  for  the  CorDs.  Performance  Measurement 
Svstem 

The  evaluation  strategy  the  authors  recommed  for  the 
perfomance  measurement  system  developed  in  the  current 
research  effort  is  a  three-phase  evaluation  strategy.  The 
reason  for  the  three  phased  rather  than  a  two  phased  approach 
is  that  the  current  system  actually  represents  a  measurement 
concept  rather  than  a  fully  developed  measurement  system. 
The  measurement  concept  requires  further  evaluation  before 
the  authors  recommend  full-scale  development  of  the 
measurement  system  iteself. 

Validation  of  the  Measurement  Concept 

The  evaluation  of  the  viability  and  validity  of  the 
measurement  concept  has  already  been  initiated.  The  first 
two  steps  taken  in  this  effort  were  the  review  of  the  concept 
by  knowledgeable  military  SMEs  with  the  appropriate  corps  and 
EAC  experience  and  the  application  of  the  synchronization 
measurement  concept  to  the  Inchon  Landing.  The  next  step  in 
the  evaluation  of  the  measurement  concept  should  be  an 
assessment  of  the  feasibility  and  adeauacy  of  the  measurement 
concept  based  on  first-hand  observations  of  the  REDCOM  Bold 
Venture  87  exercise  by  a  team  of  researchers  and  military 
SMEs  familiar  with  the  measurement  concept.  This  assessment 
will  be  highly  sujbective  in  nature  but  will  provide  valuable 
insights  for  the  development  of  the  measurment  system  itself/ 
assuming  that  the  concept  is  judged  to  be  valid. 

The  research  team  should  develop  a  set  of  specific 
questions  to  be  answered  by  their  observations  and  interviews 
conducted  with  players  and  controllers  in  the  Bold  Venture  87 
exercise.  The  questions  to  be  answered  should  cover  issues 
such  as: 

1.  Are  the  proposed  data  collection  elements  available  for 

collection  in  the  Bold  Venture  exercise? 
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2.  Does  the  measurement  concept  exclude  important 
performance  dimensions  or  factors  which  appeared  to 
impact  significantly  on  the  outcome  of  the  Bold  Venture 
exercise? 

3.  Are  the  reports  and  data  elements  anticipated  as 
available  on  JESS  actually  available? 

4.  How  many  observers  with  what,  types  of  qualifications 
would  have  been  required  to  collect  the  information 
identified  in  the  performance  measurement  concept? 

5.  Is  it  possible  to  construct  the  planned  battlefield  event 
and  staff  action  timelines  based  on  the  corps  operations 
plan  and  other  available  data? 

6.  Does  is  seem  feasible  that  a  consise  statement  of  the 
commander's  intent  and  concept  of  the  ooeration  could 
be  develooed  to  serve  as  the  basis  for  develoDment  of 
objective  auestions  used  to  assess  subordinate  under¬ 
standing  of  intent? 

7.  Does  it  appear  to  be  feasible  to  interact  with  the 

exercise  players  in  a  manner  that  would  allow  collection 
of  answers  to  questions  regarding  player  understanding 
of  the  commander's  intent  and  concept  of  the  operation? 

8.  Is  the  data  available  on  JESS  to  construct  the  actual 
battlefield  event  timeline  called  for  in  the  measurement 
concept? 

9.  Does  it  seem  feasible  that  observers  could  make  the 

necessary  observations  to  collect  data  to  construct  the 
actual  staff  action  timeline  called  for  in  the 

measurement  system? 

10.  Are  there  opportunities  to  insert  probes  into  the 
exercise  as  it  is  currently  structured  and  conducted? 

While  the  list  of  questions  above  is  certainly  not 
exhaustive#  it  does  provide  an  illustration  of  the  type  of 
information  which  should  be  collected  to  evaluation  the 
feasibility  of  the  measurement  concept  itself.  If  the  data 
collected  indicate  that  the  measurement  concept  is  viable# 
the  necessary  revisions  to  the  concept  suggested  by  the 
REDCOM  observations  should  be  made  and  a  prototype  version  of 
the  actual  measurement  system  should  be  fully  developed  for 
implementation  in  the  next  REDCOM  exercise  using  JESS 
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(probably  Bold  Eagle  88).  At  this  next  exercise  *  the 
formative  evaluation  or  phase  two  of  the  evaluation  process 
should  be  implemented. 

Formative  Evaluation  of  the  Performance  Measurement  System 

During  the  formative  evaluation  of  the  measurement 
system#  one  segment  of  the  research  team  should  be  devoted 
entirely  to  evaluation  of  the  measurement  system  as  opposed 
to  collecting  data  using  the  system!  The  temptation  usually 
exists  to  avoid  using  limited  personnel  assets  for  evaluation 
of  the  measurement  system  itself.  Instead#  the  typical 
approach  is  to  use  the  data  collectors  as  the  measurement 
system  evaluation  team.  The  problem  with  this  latter 
approach  is  that  data  collectors  often  become  highly  involved 
in  the  data  collection  process  and  are  unable  to  step  back 
and  objectively  evalute  the  measurement  svstem  itself.  As 
data  collectors,  their  attention  is  focused  on  the  evaluation 
of  the  oerformance  of  the  organization  thev  are  observing. 

The  general  nature  of  the  issues  to  be  examined  during  a 
formative  evaluation  of  a  oerformance  measurement  system  were 
noted  previously.  In  the  case  of  a  corps  performance 
measurment  system#  the  following  is  a  sample  of  the  issues 
which  should  be  carefully  examined: 

1.  To  what  extent  are  copies  of  operations  plans#  reports# 
and  other  necessary  documents  needed  for  development  of 
planned  battlefield  and  staff  action  event  timelines 
readily  made  available  to  the  research  team. 

2.  Are  the  probes  inserted  into  the  exercise  simulation  as 
planned?  If  not#  what  problems  prevented  their 
insertion? 

3.  Are  the  observers  alerted  as  planned  to  the  insertion  of 
the  probe? 

4.  Are  observers  able  to  make  the  observations  of  target 
staff  elements  as  planned  following  the  insertion  of  the 
probe? 

5.  Is  it  possible  to  acquire  the  snaoshots  of  the  battle¬ 
field  situation  from  JESS  as  planned?  What  percentage 
of  planned  snapshots  are  collected?  Is  this  an  adeauate 
number#  too  many#  an  appropriate  amount?  If  problems 
exist  in  obtaining  the  snapshots#  are  they  hardware# 
software#  or  controller  cooperation  problems? 
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6.  Is  it  feasible  to  obtain  selected  hardcopy  reports  in  the 
form  of  printouts  from  local  workstations?  Do  the 
controllers  operatinq  the  workstations  consider  this  as 
a  burden  on  their  time?  Are  there  too  many  printouts 
for  their  practical  use  of  the  data  they  contain? 

7.  Do  the  observers  appear  to  influence  the  actions  of  the 
players  in  any  adverse  manner?  Were  their  any  player 
complaints  concerning  observers  interfering  with  the 

•corps  staff  operations? 

8*  Did  the  players*  particularly  the  corps  commander* 
consider  the  feedback  given  from  the  performance 
measurement  system  as  useful? 

9.  Are  the  division  commanders  and  principal  corps  staff 
officers  available  and  willinq  to  answer  Questions 
concernina  their  understandina  of  the  corps  commander's 
conceDt  of  the  operation  and  intent?  Should  this  be 
done  verballv  or  with  a  short  questionnnaire?  Are  the 
players  defensive?  Do  the  players  consider  this  to  be 
interfering? 

The  process  by  which  the  data  will  be  collected  to 
conduct  the  formative  evaluation  of  the  measurement  process 
includes  three  data  collection  techniaues:  observation, 

interviews,  and  surveys.  The  observations  should  be  made  bv 
those  members  of  the  research  team  assianed  to  evaluation  of 
the  Derfor mance  measurment  system.  These  individuals  will 
"float"  throughout  the  exercise  and  observe  and  talk  to  data 
collectors*  controllers*  and  to  a  limited  extent*  players  in 
the  exercise.  Their  attention  should  be  focused  on  the 
actions  of  the  controllers  and  data  collectors*  not  the 
performance  of  the  players.  Two  researchers  will  probably  be 
adequate  for  performing  this  observation  role.  A  maior  focus 
of  this  observation  should  be  to  evaluate  procedures  used  by 
data  collectors  and  problems  they  experience  to  identify  the 
need  for  changes  in  observation  guides  and/or  changes  in  the 
training  of  data  collectors. 

The  second  procedure  for  collecting  formative  evaluation 
data  is  through  interviews.  All  data  collectors  should 
undergo  an  outprocessina  interview  at  the  end  of  the 
exercise.  The  interview  should  be  partiallv  structured  with 
ample  time  left  for  each  data  collector  to  provide  comments 
during  an  open-ended  portion  of  the  interview  session. 
Topics  to  be  covered  during  the  structure  portion  of  the 
interview  should  include  the  identification  of  problems  and 
potential  solutions  in  the  following  areas: 
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*  Use  of  the  observation  guides 

*  Problems  related  to  opportunities  for  observation 

*  Interactions  with  exercise  players 

*  Interactions  with  exercise  controllers 

m 

*  Potential  problems  related  to  exercise  interference 

*  Performance  factors  which  are  not  currently  captured 

which  appear  to  be  important 

*  Recommendations  for  changes  in  data  collector  training 

In  addition  to  the  interviews  with  the  data  collectors  a 
sample  of  exercise  controllers  should  also  be  interviewed. 
The  controllers  selected  for  t*ie  interviews  should  be  chosen 
from  those  individuals  who  had  extensive  interactions  with 
data  collectors  or  those  who  were  involved  in  other 
procedures  related  to  extracting  data  from  JESS  for  the 
performance  measurement  system.  The  focus  of  the  interviews 
with  the  controllers  should  be  to  determine  the  extent  to 
which  the  performance  measurement  procedures  or  data 
collector  actions  were  perceived  as  interfering  with  either 
the  controller's  job  or  the  exercise  itself.  Efforts  should 
be  made  to  solicit  controller  suggestions  for  ways  to  improve 
the  data  collection  process. 

The  final  group  of  interviews  should  be  conducted  with  a 
sample  of  players  from  the  corps.  The  individuals  selected 
for  these  interviews  should  include  individuals  who  answered 
questions  concerning  their  understanding  of  the  corps 
commander's  intent/  individuals  in  staff  elements  under 
observation  by  the  observers  associated  with  the  performance 
measurement  process/  and  individuals  who  participated  in 
feedback  sessions  based  on  the  performance  measurement  system 
data.  The  interviews  should  be  desioned  to  obtain 
information  concerning  perceived  interference  by  the 
measurement  procedures  or  observers/  suggestions  for 
improving  the  data  collection  process/  perceived  utility  of 
the  feedback/  and  any  concerns  experienced  by  the  players 
concerning  the  manner  in  which  the  performance  data  might  be 
used  by  the  Army. 

The  final  methodology  which  might  be  used  to  collect 
data  for  the  formative  evaluation  would  be  written  surveys. 
The  gains  to  be  obtained  from  conducting  a  written  survey 
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will  be  determined  by  the  size  of  the  sample  of  controllers 
and  players  who  are  interviewed.  If  time  or  other  factors 
limit  the  number  of  interviews  to  a  relatively  small  sample 
of  controllers  and  players  with  relevant  information#  then 
the  expense  of  a  mailed  survey  instrument  may  be  justified. 
The  survey  instrument  would  cover  many  of  the  same  topics 
mentioned  in  the  discussion  of  the  controller  and  player 
interviews.  The  written  survev  may  also  orovide  a  means  to 
collect  data  from  the  controllers  and  plavers  after  they  have 
had  a  chance  to  better  evaluate  their  experience  during  the 
REDCOM  exercise.  Some  change  in  perspective  from  the  period 
immediately  following  the  exercise  to  a  time  two  or  more 
weeks  later  is  likely  to  occur#  particularly  for  the 
individuals  who  were  players  in  the  exercise.  Of  particular 
importance  would  be  changes  in  the  players'  perspecitives 
regarding  the  utility  of  the  feedback  received  from  the 
performance  measurement  system. 

One  note  of  caution  which  must  be  voiced  concerning  such 
a  survey  is  that  it  is  likely  to  have  a  relatively  low  return 
rate.  This  is  likely  to  occur  reqardless  of  the  efforts  made 
by  the  research  team. 

The  data  gathered  from  all  of  the  sources  should  be 
summarized  by  issue  and  used  to  examine  all  relevant 
components  of  the  corps  performance  measurement  system.  The 
suggestions  for  revision  should  be  considered  carefully. 
Particular  attention  should  be  paid  to  indications  that  any 
aspect  of  the  performance  measurement  process  interfered  with 
the  conduct  of  the  exercise  or  the  ability  of  exercise 
controllers  to  perform  their  jobs.  Appropriate  revisions  to 
the  performance  measurement  system  should  be  made  based  upon 
the  formative  evaluation. 

If  the  formative  evaluation  results  in  relatively  minor 
changes  to  the  performance  measurement  system#  then 
performance  data  collected  during  the  exercise  may  be  used 
for  the  summative  evaluation  also.  The  summative  evaluation 
which  is  the  third  phase  of  the  evaluation  process  will  be 
designed  to  begin  with  the  same  exercise  as  the  formative 
evaluation.  If  the  formative  evaluation  results  in  major 
revisions  to  the  performance  measurement  system#  the  data 
collected  for  summative  evaluation  purposes  will  be 
disregarded  and  the  summative  evaluation  will  begin  with  the 
next  exercise. 

SUMMATIVE  EVALUATION 

As  noted  earlier#  the  summative  evaluation  of  the 
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oerformance  measurement  svstem  reoresents  what  most 
researchers  consider  to  be  the  validation  of  the  measurement 
svstem.  The  six  criteria  which  miaht  be  used  in  the 
summative  evaluation  of  a  corps  performance  measurement 
system  were  presented  previously.  As  noted  in  the  previous 
section/  some  of  this  data  can  be  collected  during  the  same 
exercise  in  which  the  formative  evaluation  is  conducted. 
This  is  particularly  true  regarding  the  collection  of  data  on 
player  reactions  to  the  feedback  system.  In  fact/  a  written 
survey  of  all  players  receiving  feedback  from  the  performance 
measurement  system  is  recommended  for  purposes  of  collecting 
data  for  summative  evaluation  purposes. 

Data  regarding  the  reliability  of  the  performance 
measurements  system  can  also  be  collected  during  the  same 
exercise  used  for  the  formative  evaluation.  Two  types  of 
reliability  data  should  be  collected.  The  first  type  of  data 
relates  to  the  reliability  of  the  event  and  outcome  data 
generated  by  JESS.  JESS  is  designed  to  allow  restarting 
during  the  exercise  itself.  It  can  be  anticipated  that  the 
computer  system  may  "crash"  at  least  once  if  not  several 
times  during  the  duration  of  an  exercise.  Such  crashes  of 
the  system  provide  a  means  to  examine  the  variablility  (test- 
retest  reliability)  of  the  effects  produced  by  the  algorithms 
used  in  JESS.  This  examination  of  reliability  is  possible 
because  the  JESS  will  essentially  restart  from  a  period  0  to 
30  minutes  prior  to  the  computer  crash  and  replicate  all 
commands  entered  by  controllers  into  the  system.  Any 
variation  in  the  status  of  units/  etc.  between  the  point  at 
which  the  model  crashed  and  the  point  at  which  it  is  ready  to 
begin  live  play  again  should  only  be  attributible  to 
variation  in  the  probablistic  algorithms  used  to  calculate 
casualties/  weapon  kills/  and  certain  other  outcomes  in  the 
model.  It  is  anticipated  that  this  variation  should  be 
relatively  minor. 

The  second  type  of  reliability  to  be  examined  in  the 
corps  performance  measurement  system  is  interrater 
reliability  for  observational  data.  The  collection  of  such 
reliability  data  will  require  the  use  of  multiple  observers 
examining  the  same  events.  This  interrater  reliability  data 
should  be  collected  during  both  Bold  Eagle  88  and  the  next 
REDCOM  exercise.  Adequate  data  should  be  gathered  to  test 
the  interrater  reliability  of  all  major  types  of  observation 
guides  used  in  the  performance  measurement  system.  The 
analysis  will  be  conducted  using  correlation  coefficients  to 
examine  the  dearee  of  consistency  in  observer  ratinas/ 
classifications/  events  recorded,  etc.  Averaae  interrater 


9-11 


reliability  coefficients  should  probably  exceed  .80  for  most 
of  the  data  collection  instruments  used.  Lower  interrater 
reliability  coefficients  may  be  a  sign  of  poor  observer 
training  or  vague  or  overly  complex  data  collection  and 
recording  procedures. 

The  evaluation  of  the  validity  of  the  performance 
measurement  system  is  likely  to  be  extremely  difficult.  As 
noted  earlier,  the  system  contains  multiple  performance 
measures  which  should  be  correlated  to  indepent  measures  of 
corps  performance  outsided  of  the  RELCOM  exercise 
environment.  Unfortunately,  the  malor  impetus  for  the 
development  of  the  current  corps  performance  measurement 
system  is  the  recoanition  of  a  lack  of  valid  measures  of 
corDs  performance.  Therefore*  it  is  unlikelv  that  a 
criterion-related  validation  strateav  would  be  aDDrooriated 
for  validatina  the  corps  performance  measurement  system  as  a 
whole . 


It  may  be  possible  to  demonstrate  at  least  internal 
validity  of  the  performance  measurement  system  bv  examinina 
the  correlations  between  different  measures  of  performance 
generated  by  the  system.  This  represents  a  construct 
validity  approach  to  validating  the  performance  measures. 
For  example,  the  performance  model  presented  in  Figure  4-3 
and  knowledae  gained  from  previous  work  with  corps  level 
oraanizations  will  allow  one  to  predict  that  certain  tvpes  of 
relationships  should  exist  between  different  measures  of 
corps  performance.  To  the  extent  there  is  consensus  amona 
knowledaable  militarv  SMEs  reaardina  the  nature  of  these 
relationships,  one  can  examine  the  extent  to  which  data 
generated  by  certain  compenents  of  the  performance 
measurement  system  are  consistent  with  what  is  to  be 
expected.  The  point  of  caution  which  must  be  voiced  at  this 
point  is  twofold.  First,  the  "validity"  of  this  approach  is 
dependent  on  the  extent  to  which  the  relationships  postulated 
bv  the  SMBs  are  valid.  Second*  onlv  certain  measures  of 
performance  can  be  validated  usina  this  process  because  it 
reauires  multiple  data  points.  Thus,  correlations  can  be 
calculated  only  for  measures  of  peformance  which  provided 
multiple  data  points  during  the  exercise.  Corps  level 
summary  measures  of  performance  cannot  be  validated  using 
this  approach,  although  some  of  the  raw  data  or  subordinate 
measures  of  performance  used  to  derive  the  summary  measure 
may  be  candidates  for  validation. 

As  noted  earlier,  the  most  likely  process  for  validating 
the  corps  performance  measurement  system  will  be  through  the 
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content  validation  process.  This  will  involve  systematic 
evaluation  by  qualified  SMBs.  The  SNEs  must  first  examine 
the  domain  of  critical  performance  factors  representative  of 
corps  level  organizations.  After  documenting  this  domain  of 
tasks*  functions*  or  performance  dimensions*  the  SMBs  must 
then  examine  the  degree  to  which  the  corps  performance 
measurement  system  adequately  samples  the  domain.  Judgments 
must  be  made  as  to  the  extent  to  which  the  measurement  system 
adquately  samples  from  the  entire  content  domain  of  corps 
performance  factors.  Such  a  judgment  may  be  quantified  in  a 
simplistic  way  such  as  indicating  the  percentage  of  the 
performance  factors  in  the  corps  performance  domain  which  are 
sampled  by  the  measurement  system.  Concommitently*  the  SMBs 
must  assess  the  extent  to  which  the  measurement  system 
includes  performance  factors  not  relevant  to  the  domain  of 
corps  performance.  These  factors  represent  sources  of 
criterion  contamination  and  should  be  removed  from  the 
system.  More  detailed  procedures  for  conducting  such  content 
analyses  are  available  in  various  publications  in  the  areas 
of  Industrial  and  Organizational  Psychology. 

Another  criterion  which  can  be  applied  to  the  evaluation 
of  the  corps  performance  measurement  system  is  the  extent  to 
which  there  is  variability  among  different  units  on  the 
performance  measures.  For  the  corps  level  summary  measures* 
this  will  require  the  collection  of  data  across  multiple 
REDCOM  exercises  and*  therefore*  will  require  considerable 
time.  On  the  other  hand*  some  of  the  raw  performance  data 
and  some  of  the  diagnostic  process  measures  and  selected 
engagement  outcome  measures  will  orovide  multiDle  data  Doints 
in  a  sinale  exercise.  Adeauate  data  to  examine  variabilitv 
on  these  measures  mav  be  obtained  from  no  more  than  two 
RBOCOM  exercises.  The  most  appropriate  measure  of  such 
variablility  is  likely  to  be  calculation  of  the  least  squares 
measure  of  variance  or  standard  deviation  of  scores  on  each 
performance  measure. 

The  cost  and  ease  of  use  of  the  corps  performance 
measurement  system  can  be  assessed  during  the  formative 
evaluation  phase  of  the  evaluation  process.  The  cost  factors 
to  be  considered  in  the  corps  performance  measurement  system 
are  primarily  related  to  the  number  of  observers  required  and 
the  types  of  skills  required  by  the  obervers.  Costs  of 
observer  training  and  costs  related  to  data  storage*  hardware 
(electronic  clipboards)*  software  (modifications  in  JESS)* 
and  development  of  observation  guides  tailored  to  each 
exercise  must  also  be  considered.  Unfortunately*  it  may  be 
extremely  difficult  to  calculate  cost  savings  or  gains  which 
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will  be  obtained  by  application  of  the  performance 
measurement  system.  This  may  preclude  a  quantitative  cost- 
benefit  analysis  unless  certain  baseline  assumptions 
regarding  benefits  can  be  made. 

The  final  criterion  measure  which  should  be  used  to 
evaluate  the  corps  performance  measurement  system  is  to 
determine  the  extent  to  which  Derformance  feedback  imoacts 
positively  on  subseauent  performance  of  the  corps.  As 
currently  structured.  it  will  be  verv  difficult  to 
demonstrate  positive  impacts  on  corns  performance  within  the 
context  of  the  REDCOM  exercise  oroaram.  The  corps  is  likelv 
to  receive  feedback  at  the  end  of  the  exercise  and  bv  the 
time  the  corns  participates  in  the  next  REDCOM  exercise  it  is 
likelv  to  have  a  new  commander  and  have  underaone  a  number  of 
other  chanaes  which  make  it  difficult  to  assess  the  impact  of 
performance  feedback.  On  the  other  hand,  followina  the  Bold 
Venture  87  exercise/  all  Army  corps  will  be  given  the 
software  and  hardware  to  practice  using  the  JESS  in  corps  and 
division  CPXs.  It  may  be  possible  to  construct  appropriate 
research  designs  to  examine  the  impact  of  a  corps  performance 
measurement  system  implemented  within  the  context  of  repeated 
uses  of  JESS  driven  exercises  within  individual  Army  corps. 

Summary 


The  general  conclusion  which  can  be  drawn  regarding  the 
validation  of  the  corps  performance  measurment  process  is 
that  it  will  be  a  difficult  procedure  which  will  require 
substantial  time.  While  certain  evidence  regarding  the 
utility  and  validity  of  the  performance  measurement  system 
may  be  obtained  in  one  or  two  runs  of  the  REDCOM  exercise 
system/  much  of  this  data  will  be  relatively  subjective  in 
nature.  More  empirical  validation  evidence  will  require  data 
collection  across  a  number  of  exercises  conducted  by  REDCOM 
and/or  by  individual  Army  corps. 

It  i 8  also  important  to  note  that  data  from  validation 
studies  may  produce  more  questions  than  answers.  If  the  data 
indicate  only  limited  evidence  for  the  validity  of  the 
performance  measures/  a  number  of  questions  must  be 
addressed.  First/  it  should  be  recognized  that  validity  is 
not  a  property  inherent  in  a  measurement  instrument.  The 
concept  of  validity  involves  the  purpose  for  which  data 
produced  by  the  measurement  instrument  are  to  be  used.  For 
example/  the  measurement  of  a  particular  staff  process  may 
not  be  significantly  related  to  measures  of  synchronization 
on  the  battlefield  but  may  be  moderately  correlated  to  other 
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staff  actions  that  are  related  to  synchronization.  Putting 
aside  the  issue  of  determination  of  causality  in 
correlational  data/  the  issue  of  the  validity  of  the  measure 
of  performance  will  depend  on  whether  one  is  interested  only 
in  performance  measures  directly  related  to  battle  outcomes 
or  in  measures  of  performance  which  may  indirectly  impact  on 
battle  outcomes  through  their  impact  on  other  staff 
performance  factors. 

Furthermore/  data  indicating  low  correlations  between 
measures  may  indicate  any  of  a  number  of  problems.  If  the 
exercise  or  simulation  produces  relatively  little  variation 
between  units  or  across  multiple  runs  with  the  same  unit/  it 
will  be  statistically  impossible  to  demonstrate  validity  of 
the  performance  measures  and  the  problem  may  be  in  the 
simulation;  not  tl  >  performance  measurement  system.  If  the 
simulation  produces  variation  in  performance  but  the  measures 
do  not  appear  to  be  sensitive  to  the  performance  variation 
then  at  least  two  possiblities  exist.  First/  the  measurement 
procedures  may  be  inadequate  and  the  performance  factors  may 
need  to  be  operationalized  in  a  different  manner. 
Alternatively/  it  may  be  the  case  that  the  performance  factor 
and  model  of  performance  develooed  in  step  three  of 
developing  the  performance  measurement  system  is  inaccurate 
and  new  performance  factors  must  be  identified.  The  answers 
to  such  auestions  will  not  be  directlv  provided  bv  the 
validation  data.  Instead/  thev  will  reauire  careful 
consideration  of  the  validation  data  and  other  information  bv 
an  exDerienced  researcher  with  extensive  knowledqe  of  the 
performance  measurement  system  and  the  nature  of  the 
organizational  element  and  operations  being  assessed  by  the 
measurement  system. 
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APPENDIX  A 


QUESTIONS  USED  IN  REDCOM  INTERVIEWS 


3. 


Structure  of  Bold  Venture  87  Exercise 


Exact  dates  for  JTLs  play 
Exact  dates  for  play  with.  JESS 
Army  units  involved 
Air  Force  Units  involved 

Schedule  of  key  exercise  (not  scenario)  events 

Location  of  all  players  and  controllers 

Debriefing/  After  Action  Review  Sessions 

Interplay  between  JTLs  and  JESS 

Ideally,  would  like  operations  plan  for  exercise 

How  similar  will  future  JVC  exercises  be  to  Bold  Venture 
87?  How  will  they  differ? 

Exercise  Scenario 


Ideally,  would  have  copy  of  scenario 

Need  enough  detail  to  identify  potential  probe  events 

Need  overview  and  background  of  the  scenario  if  we  cannot 
obtain  a  complete  copy 

Copies  of  messages  generated  by  JTLs  and  JESS  which  are 
used  to  drive  the  exercise 


Senior  Controller  Functions 

■  Who  will  act  as  Senior  Controller? 

■  Written  description  of  role  if  available 

■  How  does  the  Senior  Controller  and  his  staff  relay 
information  to  players? 
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Who  controls  the  senior  controller?  Or  is  he  "God”? 

What  is  the  degree  of  potential  influence  of  activities 
and  outcomes  exercised  by  the  senior  controller? 

What  is  the  relationship  (during  the  exercise)  of  the 
senior  controller  and  the  Corps  Commander? 


What  is  the  relationship  of  the  senior  controller  and  the 
Joint  Task  Force  Commander? 

How  accurately  do  controller/players  provide  appropriate 
information  to  players  during  the  exercise? 

What  is  the  senior  controller's  role  in  the  real  world  vs. 
in  the  exercise? 

Controller/Interfacers  vs.  Controller/Players? 

What  is  the  relationship  of  the  senior  controller  and  and 
the  OPFOR? 

Does  the  OPFOR  have  access  to  all  information? 

What  training  for  role  does  the  senior  controller 
currently  receive? 

What  data  collection  or  recording  is  done  by  the  senior 
controller  or  his  staff? 

What  is  the  primary  focus  of  the  senior  controller's 
attention? 

What  is  the  senior  controller's  role  in  AARs? 


Opportunities  for  Observation  by  ARI  Data  Collectors 

i  Where  are  the  nodes  that  observers  could  be  placed? 

I  Who  (rank,  specialties,  etc.)  will  act  as  observers  for 
REDCOM?  How  many? 


■  From  what  REDCOM  element  (J-3.  JWC,  or  J-5)  are  the 
observers  coming? 

■  Where  will  REDCOM  observers  be  placed? 

■  Will  the  Army  and  Air  Force  units  in  the  exercise  have 
additional  observers?  Who?  How  many?  Where? 
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■  What  Corps  staff  elements  will  be  observed  most  closely  by 
REDCOM? 

■  Will  REDCOM  observers  have  any  checklists,  data  collection 
forms,  etc.? 

■  Is  REDCOM  providing  any  training  for  their  observers? 

■  Are  there  any  particular  events  in  the  scenario  that  are 
being  used  to  focus  observer  attentions  (probes)? 

5.  Potential  Probe  Events 

■  How  could/would  probe  events  be  implemented? 

■  Are  any  probe  events  currently  planned? 

■  How  could  observers  be  alerted  to  occurrence  of  the  probe? 

■  Can  probe  events  be  recorded  and  activities  generated  by 
the  probe  tracked  automatically  by  JESS? 

6.  Operation  of  JESS  as  an  Exercise  Driver 

■  Sample  messages,  screen  displays,  etc. 

■  What  is  the  interface  between  JESS  and  JTLs? 

■  What  tracking  capabilities  and  data  extraction 

capabilities  exist  in  JESS  to  allow  tracking  of  player 

activities? 

■  What  data  if  any  will  be  extracted  and  saved  from  the 
exercise  run  as  currently  planned? 

■  What  medium,  if  any,  can  be  used  for  saving  data  from 

JESS? 

■  Exactly  how  are  player  responses  entered  into  JESS? 

■  What  is  lag  time  between  a  response  by  a  player  such  as 
preparing  an  order  or  message  and  the  time  that  response 
enters  the  computer  system  and  is  recorded  by  JESS?  How 
close  to  real-time? 

■  Are  responses  tagged  to  specific  players  (corps  staff 

elements)  or  to  controllers  who  enter  response  on  the 
system? 

■  Do  input  restrictions  for  JESS  limit  player  actions  or 

creativity  in  any  way? 
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■  Is  it  possible  to  extract  intermediate  performance 
indicators  from  JESS?  These  are  major  actions  such  as 
movement  of  Divisions ,  execution  of  air  strikes,  etc. 

■  How  will  the  interface  between  the  Army  and  Air  Force  be 
played  in  JESS? 

■  What  is  current  assessment  of  validity  of  battle  outcomes 
generated  by  JESS? 

■  Is  it  possible  to  re-establish  a  sequence  of  orders  and 
commands  in  JESS?  For  how  long? 

■  Is  the  role  of  JESS  to  generate  exercise  realism  or  battle 
outcomes?  Which  is  primary? 

7.  Role  of  JTLs  and  its  Interface  with  JESS 

■  How  is  JTLs  being  used  for  the  build  up  for  Bold  Venture 
87? 

■  What  is  the  corps  player  involvement  in  the  pre-exercise 
build  up  phase?  Why? 

■  What  players  from  the  participating  corps  are  involved? 

■  How  do  JESS  and  JTLs  interface  with  respect  to  the 
architecture  of  the  two  models? 

■  How  are  the  JTLs  and  JESS  databases  related  or  linked? 

■  How  different  are  the  algorithms  used  in  JTLs  and  JESS  to 
produce  battle  outcomes? 

■  Is  JTLs  used  strictly  to  provide  Intelligence  situation 
development? 

■  Are  players  allowed  to  impact  on  situations  in  JTLs,  i.e., 
are  they  moving  and  fighting  maneuver  elements? 
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APPENDIX  B 


REVIEW  OF  PERFORMANCE  MEASUREMENT  SYSTEMS 


General 

The  review  of  the  performance  measurement  literature 
indicates  that  the  information  processing  model  of  corps 
performance  is  consistent  with  the  models  used  in  previous 
attempts  to  measure  performance  of  commanders  and  their 
staffs.  The  review  of  existing  measurement  systems 
developed  to  assess  performance  of  Army  staffs  provides 
evidence  supporting  the  information  processing  approach 
adopted  in  the  current  effort.  The  review  also  identifies  a 
number  of  problems  which  must  be  avoided  in  development  of 
the  current  performance  measurement  system. 

FORGE  Research  Program 

A  number  of  research  studies  have  been  conducted 
examining  unit  performance.  Many  of  these  studies  have 
focused  on  evaluation  of  performance  at  the  battalion  level. 
One  group  of  studies  particularly  relevant  to  the  current 
effort  is  a  series  of  studies  which  are  commonly  labeled  as 
the  FORGE  research  studies.  This  research  was  conducted  by 
Olmstead  and  his  associates  (Olmstead,  Christensen,  and 
Lackey,  1973;  Olmstead,  Baranick,  and  Elder,  1978;  Olmstead, 
Elder,  and  Forsyth,  1978)  in  the  1970's.  The  research 
focused  on  the  measurement  of  information  processing  of 
battalion  staffs  and  the  impact  of  organizational  processes 
on  battalion  performance  in  combat  simulation  exercises. 

The  information  processing  model  used  by  Olmstead  is 
based  on  seven  processes  derived  from  Schein's  (1972) 
organizational  Adaptive  Coping  Cycle.  These  seven 
organizational  processes  and  their  definitions  are  presented 
in  Table  B-l.  The  seven  organizational  processes  were 
assumed  to  be  related  to  the  organizational  competence  of  a 
battalion.  Organizational  competence  is  defined  in  terms  of 
three  components  which  Olmstead,  Elder,  and  Forsyth  (1978) 
defined  as: 

Reality  Testing.  Capacity  to  assess  the  reality  of 
situations  facing  the  organization  —  the  ability  of  the 
organization  to  search  out,  accurately  perceive,  and 
correctly  interpret  the  properties  and  characteristics  of 
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TABLE  B— 1.  DEFINITIONS  OF  ORGANIZATIONAL  PROCESSES 


Sensing 

The  process  by  ehich  an  organization 
acquires  and  processes  information  about 
its  internal  and  external  environments. 

Communicating 

Information 

The  process  of  transmitting  information 
that  is  sensed  to  those  parts  of  the 
organization  that  can  act  upon  it. 

Decision  - 
flaking 

The  process  of  making  decisions  concerning 
actions  to  be  taken  as  the  result  of 
sensed  information. 

Stabalizing 

The  process  of  taking  actions  to  adjust 

Internal  functioning  and  maintain  organ!' 
zational  stability  and  integration  that 
might  otheraise  be  disrupted  as  a  conse¬ 
quence  of  actions  taken  to  cope  oith 
changes  in  the  organization's  environments. 

Communicating 

Implementation 

The  process  of  transmitting  decisions  and 
decisions-related  orders  and  instructions 
to  those  parts  of  the  organization  that 
must  implement  them. 

Coping  Actions 

The  process  of  executing  actions  intended 
to  cope  eith  changes  in  the  organization's 
environments. 

Feedback 

The  process  of  evaluating  the  results  of  a 
prior  action  through  futher  sensing  of  the 
external  and/or  internal  environments. 
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its  environments  (both  external  and  internal) ,  particularly 
properties  that  have  relevance  for  the  objectives  and 
survival  of  the  organization. 

Adaptability.  The  capacity  of  the  organization  to  solve 
problems  arising  from  changing  environmental  demands  and  to 
act  with  effective  flexibility  in  response  to  these  changing 
demands . 

Integration.  The  maintenance  of  structure  and  internal 
functions  under  change  and  stress,  and  a  state  of  relations 
among  sub-units  that  insures  that  coordination  is  maintained 
and  sub-units  do  not  work  at  cross  purposes. 

In  one  of  the  later  studies  conducted  as  part  of  this 
programatic  research  effort,  a  measurement  system  based  on 
the  adaptive  coping  cycle  model  was  used  to  examine  the 
relationship  between  battalion  information  processing 
effectiveness  and  battalion  performance.  The  study  was 
performed  as  part  of  the  Cardinal  Point  II  exercises 
conducted  in  July  to  August  in  1978.  The  measurements  were 
obtained  during  the  battle  simulation  phase  of  the  Cardinal 
Point  II  exercises  using  the  PEGASUS  simulation. 

Olmstead,  et.al.  used  trained  Organizational 
Effectiveness  Staff  Officers  to  make  subjective  ratings  on 
measures  of  each  of  the  seven  process  dimensions  defined  in 
Table  B-l.  The  observers  provided  ratings  on  a  single  four 
point  scale  for  each  of  the  seven  dimensions.  The  OE  staff 
officer  ratings  on  the  seven  organizational  process  measures 
were  then  correlated  with  controller  ratings  of  battalion 
effectiveness  during  the  PEGASUS  simulation.  Olmstead,  et. 
al.  found  highly  significant  correlations  between  the 
ratings  on  organizational  process  measures  and  ratings  of 
battalion  effectiveness  in  the  simulation. 

While  the  significant  correlations  obtained  by 
Olmstead,  et.  al.  suggest  that  an  information  processing 
model  of  battalion  effectiveness  has  utility,  several  notes 
of  caution  must  be  voiced.  First,  all  of  the  measures  used 
in  the  study  were  highly  subjective  in  nature.  Second,  the 
nature  of  the  simulation  provided  little  opportunity  for 
factors  other  than  information  processing  to  impact  on 
combat  outcomes.  Finally,  the  measures  used  by  Olmstead  and 
his  associates  provide  relatively  little  objective  feedback. 
The  feedback  provided  to  the  units  in  the  FORGE  research  was 
highly  qualitative  in  nature  and  any  positive  impacts 
produced  by  the  feedback  probably  reflected  the  skills  of 
the  trained  OE  officers  as  much  or  more  than  the  strengths 
of  the  measurement  system  itself. 
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While  the  FORGE  research  methodology  can  be  criticized 
for  the  reasons  cited  above,  there  has  been  renewed  interest 
in  this  research  in  the  recent  years  (McGee,  1985) .  This 
interest  was  sparked  by  observations  of  battalion 
performance  at  the  National  Training  Center.  The 
observations  have  been  interpreted  as  supportive  of  the 
findings  from  the  FORGE  research  and  prompted  action  to 
develop  new  measures  of  "battle  staff  integration."  While 
the  current  concept  has  been  labeled  as  battle  staff 
integration,  it  reflects  essentially  the  same  organizational 
process  model  used  by  Olmstead  et.al.  (1978). 

BEAT.  .Method  <?1  p.gy 

An  information  processing  model  of  command  and  staff 
performance  at  an  organizational  level  closer  to  corps  was 
developed  by  Defense  Systems,  Inc  (1978)  in  a  project 
examining  ithe  appropriate  size  for  theater  level 
headquarters.  In  early  stages  of  the  project,  it  was 
recognized  that  there  was  a  lack  of  theory  and  measurement 
procedures  for  assessing  the  effectiveness  of  theater  level 
headquarters  elements. 

The  performance  model  which  evolved  in  the  theater 
level  headquarters  research  effort  is  essentially  an 
information  processing  model.  The  headquarters  element  was 
"regarded  as  analogous  to  an  adaptive  control  system  where 
control  is  exercised  through  planning  and  execution  cycles." 
The  model  focuses  on  the  information  processing,  planning, 
and  decision  making  functions  of  headquarters  elements  as 
well  as  the  manner  in  which  the  headquarters  element  is 
structured . 

The  measurement  system  developed  for  use  in  the  study 
of  the  theater  level  headquarters  is  labeled  The 
Headquarters  Effectiveness  Assessment  Tool  (HEAT) .  This 
system  is  commonly  referred  to  as  the  HEAT  methodology.  The 
methodology  attempts  to  assess  the  quality  of  seven 
organizational  processes  which  are  listed  and  defined  in 
Table  B-2. 

The  quality  of  the  organizational  processes  is  assessed 
by  completing  12  data  forms  which  require  ratings  on  the 
quality  of  various  reports,  briefings,  messages,  etc. 
produced  by  the  headquarters  element  as  well  as  observations 
of  the  headquarters  staff  during  planning  and  execution 
phases  of  an  exercise.  Figure  B-l  illustrates  the  steps 
involved  in  implementing  the  HEAT  methodology. 

The  organizational  process  measures  derived  from  the 
ratings  of  quality  of  the  six  phases  of  the  headquarters 
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TABLE  B-2.  HEAT  METHODOLOGY  PROCESSES 


Honitor 


l  Understand 


Alternative 

Actions 


Prediction 


The  headquarters  must  obtain  raa  data  con¬ 
cerning  those  aspects  of  the  environment 
that  it  aishes  to  oontrol.  The  quality  of 
monitoring  can  be  Matured  by  directly  com¬ 
paring  the  perceptions  in  the  headquarters 
(HQ)  to  reality.  A  secondary  measure  is 
the  age  of  the  information  available  to  the 
headquarters. 

The  headquarters  processes  the  available 
information  to  produce  an  understanding  of 
the  situation.  i.e.#  a  set  of  hypotheses 
about  vhat  can  be  going  on  noa  and  in  the 
immediate  future.  The  best  understanding 
is  one  in  ehich  the  primary  hypothesis  - 
the  one  considered  most  likely  -  is  a 
reasonable  approximation  of  reality.  The 
quality  of  understanding  can  be  assessed  by 
measuring  the  correctness  of  this  set  of 
hypotheses. 

The  headquarters  develops  alternatives 
specifying  ahat  can  be  done  to  alter  the 
situation  understood  to  exist.  The  quality 
of  alternatives  can  be  measured  by: 

1)  their  completeness 

2)  their  number 

For  each  alternative  action  considered,  the 
HQ  makes  a  prediction  about  consequences. 
This  includes  at  least  tao  elements: 

1)  ahether  the  material/force  assets 
exist  or  can  be  assembled  to  carry 
out  the  alternative 

2)  ahat  the  enemy's  response  to  it  mill 
be. 

Predictions  can  be  evaluated  by  determining 
their  completeness  end  the  correctness  of 
those  predictions  adopted. 
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TABLE  B-2.  HEAT  METHODOLOGT  PROCESSES.  CONT. 


Decisions 


Direction 


Interaction 
With  the 
Environment 


Decisions  are  mode  on  the  basis  of  the 
predictions.  There  are  no  direct  measures 
of  deoision  quality  in  the  HEAT  system. 
Hooever,  decisions  aleays  take  the  form  of 
a  plan  to  be  implemented,  specifying 
missions,  opersting  boundaries,  assets,  and 
a  timetable  for  subordinate  forces. 


Plans  are  communicated  to  the  appropriate 
organizations  in  the  fora  of  some  direct* 
ive.  The  correctness  of  directives  can  be 
assessed  by  examining  the  extent  to  vhich 
the  decision  and  plan  are  correctly  stated. 


Other  important  diagnostic  measure  focus  on 
the  non-effectiveness  processes  by  vhich 
the  HQ  interacts  vith  its  environment  — 
inquiries  and  reports.  The  quality  of 
these  processes  can  be  measured  by: 

1)  the  timeliness  of  inquiries  made  by 
the  HQ  based  on  the  age  of  the  infor¬ 
mation  it  is  monitoring 

2)  the  clarity  and  completeness  of  its 
direction  to  subordinates  based  on 
their  inquiries  seeking  information, 
clarification,  end  addition 

3)  the  quality  of  the  reports  the  HQ 
produces 
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operation  cycle  are  supplemented  by  various  effectiveness 
ratings.  Unfortunately,  the  effectiveness  ratings  used  in 
the  methodology  are  also  relatively  subjective  judgments  as 
to  the  quality  of  plans  produced  by  the  headquarters 
element.  Also,  like  the  measurement  procedures  used  in  the 
FORGE  research,  the  system  focuses  primarily  on  process 
rather  than  outcome  oriented  measurement. 

While  the  HEAT  methodology  suffers  from  many  of  the 
same  problems  as  the  FORGE  research  measurement  methodology, 
it  does  have  a  number  of  strengths.  First,  as  is  suggested 
in  Figure  B-  1,  the  system  provides  an  opportunity  for  some 
tailoring  the  measurement  system  to  the  particular  exercise 
and  unit  being  evaluated.  This  feature  is  essential  at  the 
corps  level  and  above  because  of  the  relatively  unique 
missions  and  structures  of  each  organization.  Furthermore, 
the  system  does  provide  a  pool  of  potential  observation 
items  for  various  elements  found  in  higher  level 
headquarters.  While  a  process  oriented  system  is  not 
sufficient  for  a  corps  performance  measurements  system,  it 
is  one  essential  component  of  the  system  and  like  the  FORGE 
research  measurement  system,  is  focused  on  the  battalion 
level.  However,  it  is  focused  on  outcome  rather  than 
process  measurement.  The  underlying  model  for  the  NTC 
measurement  system  is  illustrated  in  Figure  B-2. 

The  NTC  performance  measurement  system  is  being 
developed  primarily  for  the  purpose  of  providing  training 
feedback  and  research  data  for  lessons  learned  development. 
The  feedback  will  be  based  upon  a  backward  audit  trail 
concept  which  begins  with  measures  of  unit  performance 
effectiveness.  These  effectiveness  measures  are  developed 
from  bottom  line  combat  outcome  data.  The  particular 
outcome  measures  and  weights  assigned  to  each  measure  depend 
on  the  particular  type  of  mission  being  conducted  by  the 
battalion  task  force.  In  addition  to  the  bottom  line 
feedback,  information  will  be  provided  on  how  well  the 
battalion  and  its  subordinate  elements  performed  mission 
critical  tasks. 

Supplementing  the  performance  measures  described  above 
will  be  observational  data  provided  by  trained  observer 
controllers  and  a  highly  objective  graphic  representation  of 
the  actual  battlefield  events  which  occurred  throughout  the 
entire  mission.  This  latter  feedback  is  made  possible 
through  the  highly  sophisticated  instrumentation  system  at 
the  NTC.  This  instrumentation  system  makes  it  possible  to 
literally  replay  a  history  tape  which  provides  positions  and 
firing  data  of  all  major  weapon  systems  during  the  entire 
mission.  As  part  of  the  after  action  review  of  each 
mission,  a  short  summary  tape  of  key  events  is  extracted 
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FIGURE  B-2.  NTC  Unit  Combat  Effectiveness  Model 


from  the  mission  history  tape  and  replayed  at  the  beginning 
of  the  review  session.  This  tape  and  the  data  and 
observations  made  by  observers  provides  a  highly  effective 
vehicle  for  providing  training  feedback. 

As  might  be  ascertained  from  the  brief  description  of 
the  NTC  performance  measurement  system,  it  is  considerably 
more  objective  than  either  the  measures  used  in  the  FORGE 
research  or  the  HEAT  methodology.  This  is  possible,  in 
part,  because  the  environment  and  instrumentation  system  at 
the  NTC  provide  very  realistic  combat  conditions  and  highly 
objective  casualty  and  battle  damage  assessment  data.  It 
should  be  noted,  however,  that  the  performance  model  and 
measurement  concepts  used  to  develop  the  unit  effectiveness 
and  critical  task  performance  measurement  systems  were 
primarily  outcome  rather  than  process  oriented. 

The  three  performance  measurement  systems  described 
above  represent  behavioral  science  approaches  to  examining 
the  performance  of  a  unit  or  organization.  Each  of  the 
approaches  involves  collecting  data  from  an  actual 
organization  or  unit  engaged  in  a  training  exercise  or 
operational  mission.  A  second,  radically  different  approach 
for  examining  performance  of  a  corps  organization  should  be 
addressed  at  least  briefly.  The  approach  referred  to  is  a 
modeling  approach.  A  variety  of  combat  simulation  models 
exist  which  are  designed  to  model  the  actions  and  outcomes 
of  various  organizational  elements  with  different 
configurations,  different  weapons  systems,  under  conditions 
of  various  force  ratios,  etc.  These  models  produce  a 
variety  of  combat  outcome  data  related  to  casualties 
suffered,  casualties  inflicted,  equipment  losses,  etc.  The 
models  have  been  used  for  a  variety  of  purposes  including 
evaluation  and  development  of  doctrinal  concepts.  The 
models  have  particular  appeal  in  examining  the  performance 
of  large  units  because  it  is  extremely  difficult  to  conduct 
realistic  maneuvers  with  corps  and  larger  elements. 

As  noted  by  DePuy  (1984)  and  others  (McEnany,  1977; 
Wood,  1982)  extreme  caution  must  be  used  in  applying  such 
models  to  predict  the  performance  capabilities  of  actual 
units.  For  the  most  part,  such  combat  models  reflect  only 
the  effects  of  weapons  systems  and  other  physical  effects 
with  little  or  no  input  reflecting  the  human  element  of 
combat.  Furthermore,  such  models  provide  limited  means  for 
a  commander  to  assess  the  effectiveness  of  his  actual 
organization.  For  these  reasons,  it  is  assumed  that  combat 
models  alone  do  not  provide  a  viable  alternative  for 
producing  corps  measures  of  performance.  On  the  other  hand, 
the  use  of  such  combat  models  as  part  of  corps  level 
exercises  is  assumed  to  be  an  essential  element  in  the 
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current  measurement  system.  Furthermore,  the  models  will 
provide  some  outcome  measures  which  represent  one  source  of 
criterion  data  to  be  used  in  the  corps  performance 
measurement  system. 

The  review  of  the  performance  measurement  literature 
and  examination  of  each  of  the  four  measurement  approaches 
briefly  described  above  suggests  several  conclusions. 
First,  there  is  evidence  that  information  processing  models 
of  command  and  staff  performance  may  be  related  to  effective 
performance  of  combat  units.  Second,  while  information 
process  measures  may  be  important,  process-oriented 
measurement  systems  tend  to  produce  highly  subjective  data 
which  is  not  outcome-oriented.  Third,  a  performance 
measurement  system  resembling  that  under  development  for  the 
NTC  has  more  potential  for  acceptability  and  utility  than  a 
performance  measurement  system  which  is  primarily  process 
oriented. 

Another  lesson  learned  by  the  present  authors  from  work 
conducted  at  the  NTC  which  is  not  discussed  above  is  that 
simply  having  large  amounts  of  objective  data  does  not 
ensure  that  commanders  and  their  staff  will  receive 
effective  feedback.  In  fact,  work  at  the  NTC  indicates  that 
an  effective  performance  measurement  system  must  be 
developed  based  on  some  model  of  unit  performance  rather 
than  developed  inductively  by  aggregating  large  amounts  of 
unrelated  data  which  has  been  collected  simply  because  an 
instrumentation  system  made  it  feasible  to  collect. 

The  review  of  the  performance  measurement  literature 
also  suggests  that  extreme  caution  must  be  exercised  in 
using  combat  outcome  measures  generated  by  combat  models. 
This  finding  reinforces  the  warning  voiced  earlier 
concerning  the  use  of  bottom  line  combat  outcome  measures  as 
the  primary  criterion  measure  for  assessing  corps  level 
performance. 
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APPENDIX  C 


THE  REDCOM  JOINT  EXERCISE  PROGRAM 


general 

As  noted  earlier,  one  assumption  made  in  the  development 
of  the  corps  performance  measurement  system  is  that  the 
system  will  be  implemented  in  the  context  of  joint  exercises 
conducted  by  REDCOM.  To  understand  the  nature  of  the 
performance  measurement  system  and  the  validation  plan  for 
the  system,  the  reader  must  have  a  basic  understanding  of 
the  manner  in  which  the  joint  exercises  at  REDCOM  are 
structured  and  conducted.  This  section  of  the  report 
provides  an  overview  of  the  structure  and  procedures 
evolving  for  the  joint  exercises  which  will  be  conducted  by 
REDCOM. 

Structure  of  a  joint  REDCOM  Exercise 

Figure  C-l  provides  a  schematic  of  the  key  components 
contained  in  a  REDCOM  exercise.  The  major  components 
consist  of  the  exercise  players,  the  Joint  Exercise  Support 
System  (JESS) ,  The  Joint  Theater  Level  Simulation  (JTLS) , 
the  exercise  controllers,  the  OPFOR  controllers,  and  a 
senior  exercise  control  element.  Each  of  the  major 
components  and  the  relationships  between  the  components  is 
described  briefly  in  the  paragraphs  which  follow. 

Exercise  Players 

All  of  the  exercises  conducted  by  REDCOM  are  designed  as 
joint  exercises.  Of  particular  interest  to  the  current 
effort  are  the  Bold  Venture  and  Bold  Eagle  exercises.  Bold 
Venture  is  a  joint  Army  -  Air  Force  exercise  while  Bold 
Eagle  is  a  joint  Army  -  Navy  exercise.  The  Bold  Venture 
exercise  is  also  strictly  a  CPX  while  Bold  Eagle  has  both 
CPX  and  FTX  components.  In  both  exercises,  the  Army  element 
participating  in  the  CPX  will  be  a  corps-size  organization. 
Beginning  with  the  Bold  Eagle  88  exercise,  the  REDCOM 
exercises  should  include  play  from  a  complete  echelon  above 
corps  (EAC)  element.  This  echelon  will  be  represented  by  a 
Joint  Task  Force  organization  commanded  by  a  four  star 
general . 

On  the  Army  side  of  the  house,  the  exercises  are  designed 
to  allow  participation  of  all  members  of  the  corps  and 
division  staff  including  the  corps  and  division  commanders. 
The  brigade  commanders  and  staff  function  in  a  player- 
controller  role  in  these  exercises.  The  division  and 
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FIGURE  0-1 .  REDCOM  Exercise  System 
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corps  staff  function  in  their  normal  manner  and  the  vast 
majority  of  these  players  have  no  direct  interface  with 
JESS,  the  computer  model  driving  the  exercise.  In  fact,  the 
workstations  are  located  in  a  centralized  building  with  most 
of  the  division  and  corps  players  deployed  in  the  field  in 
distributed  CPs. 

The  JESS  is  designed  to  enable  divisions  to  participate 
in  the  REDCOM  exercises  while  remaining  at  their  home- 
station  locations.  This  distributed  exercise  system  will  be 
tested  in  Bold  Venture  87.  Assuming  that  the  test  proves 
successful,  future  exercises  will  be  designed  to  include 
more  players  in  a  cost  effective  manner. 

The  role  of  the  Air  Force  will  be  played  by  a  small 
number  of  Air  Force  controllers  in  Bold  Venture  87.  In 
subsequent  exercises,  however,  the  Tactical  Air  Command 
(TAC)  or  other  Air  Force  elements  will  provide  players  at 
appropriate  organizational  levels.  The  use  of  a  distributed 
exercise  system  will  contribute  to  increased  participation 
of  the  Air  Force  in  future  exercises. 


JESS:  Overview  of  the  System 

JESS  is  a  computerized  battle  simulation  system.  It 
might  best  be  thought  of  as  a  computerized  battle  board  that 
eliminates  the  need  for  manual  movement  of  game  pieces  on 
map  boards.  It  also  provides  automatic  calculation  of  a 
variety  of  battle  outcome  data  which  were  previously 
calculated  manually  by  controllers.  The  reader  should  be 
cautioned,  however,  that  JESS  is  designed  primarily  as  an 
exercise  driver.  JESS  is  not  a  model  with  extensive 
analytic  or  predictive  modeling  capabilities.  JESS  is 
currently  in  the  developmental  phase  and  its  refinement  will 
continue  through  at  least  1989  or  1990. 

At  the  present  time,  there  is  no  performance  measurement 
feedback  system  built  into  JESS.  While  JESS  generates 
substantial  amounts  of  data  which  could  serve  as  part  of  a 
performance  measurement  system,  the  information  is  not 
currently  saved  by  the  system.  A  performance  measurement 
capability  on  the  JESS  is  planned  for  the  future.  The 
performance  measurement  concepts  developed  in  this  research 
effort  may  serve  as  the  basis  for  the  development  of  this 
performance  feedback  system. 

The  JESS  is  run  on  a  network  of  mini-  and  microcomputers 
with  a  central  processing  unit  actually  running  the 
simulation  program.  The  system  includes  approximately  30  to 
40  work  stations  for  a  joint  exercise 


C-3 


including  an  Army  Corps  and  appropriate  Air  Force  and/or 
Naval  units.  Each  work  station  consists  of: 


■  Two  CRT  (VT  220)  terminals  for  entering  and  displaying 
commands  and  reports 

■  A  large  color  graphics  monitor  for  display  of  unit 
locations  on  terrain  maps  and  anit  status  displays 

■  A  printer  for  obtaining  hard-copy  printouts  of  reports 

The  JESS  work  stations  are  typically  distributed  in  the 
following  manner: 

1.  At  corps  level: 

-  A  corps  fire  support  station  for  corps  artillery 

-  A  corps  combat  support  station  representing  non- 
divisional  assets  except  those  of  the  Corps  Support 
Command  (COSCOM) 

-A  corps  admin/log  station  representing  COSCOM  units 

2.  At  division  level: 

-  Brigade  stations  representing  Bde  HQs  and  maneuver 
battalions 

-  A  division  fire  support  station  for  division  art¬ 
illery 

-  A  division  combat  support  station  for  non-DISCOM 
division  support  units 

-  A  division  admin/log  station  for  DISCOM  elements 

3.  BLUEFOR  Air: 

-  Work  stations  for  Blue  tactical  air  control 

4.  For  OPFOR  controllers: 

-  Maneuver  and  fire  support  work  stations 

-  A  tactical  air  work  station 

-  A  control/log  work  station 
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5.  To  assist  in  overall  control  of  the  exercise 

-  A  theater  logistics  workstation 

-  A  Blue  ground  forces  master  control  station  manned 
by  REDCOM  exercise  controllers 

-  A  Blue  tactical  air  master  control  station  manned 
by  REDCOM  exercise  controllers 

« 

-  A  technical  master  control  station  manned  by  JPL 
computer  science  personnel 

Exercise  Controllers 

Each  JESS  workstation  is  manned  by  a  team  of  controllers 
which  may  vary  in  size  from  7  to  more  than  50  individuals 
idepending  on  the  function  of  the  workstation.  For  example, 
a  brigade  work  station  is  typically  manned  by  a  minimum  of  8 
individuals  with  the  following  composition: 

■  Team  Chief 

■  Operations  controller 

■  Intelligence  controller 

■  Fire  support  controller 

■  Administrative/logistics  controller 

■  Non-Commissioned  Officer  assistant 

■  Air  Liaison  controller 

Certain  workstations  such  as  the  division  and  corps  fire 
support  and  admin/log  stations  may  be  manned  by  considerably 
larger  teams. 

The  controllers  at  each  workstation  interface  with  JESS 
by  entering  commands  from  a  variety  of  menu  screens  used  to 
perform  various  functions.  Functions  which  may  be  performed 
at  each  workstation  vary  with  the  type  of  workstation.  At 
brigade  workstations  which  control  the  maneuver  battalions, 
controllers  may  enter  commands  to  move  units,  change  the 
task  organization  of  units  under  their  control,  call  for 
fire  support  missions,  lay  minefields,  breach  minefields, 
change  air  defense  postures  of  their  units,  and  perform 
other  functions  related  to  control  of  tactical  units  at  the 
battalion  level.  With  the  exception  of  the  controllers  at 
the  master  control  stations,  the  controllers  at  each 
workstation  can  enter  commands  which  affect  only  the  units 
under  the  control  of  the  particular  workstation  that  they 
operate.  For  example,  the  controllers  at  a  brigade  work 
station  can  control  the  movement  of  only  the  maneuver 
elements  organic  to  and  attached  to  the  particular  brigade 
they  represent . 
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The  controllers  at  the  JESS  workstations  play  a  much  more 
important  role  that  that  of  simply  entering  commands  on  a 
CRT.  The  workstation  controllers  are  the  primary  interface 
between  the  players  in  the  REDCOM  exercise  and  the  JESS. 
The  controllers  function  as  role  players  who  translate 
information  displayed  on  the  graphics  monitors  and  CRTs  of 
the  JESS  workstation  into  the  type  of  information  which 
would  normally  be  available  to  commanders  and  their  staffs 
at  the  division  and  corps  levels. 

The  role  playing  demands  become  readily  apparent  when  one 
realizes  that  all  controllers  at  workstations  can  see  the 
locations  of  all  units  (BLUE  and  RED)  involved  in  the  entire 
operation.  In  other  words,  the  controllers  at  the  work 
stations  have  perfect  information.  The  burden  of  judging 
the  amount  of  information  and  timing  for  release  of 
information  available  on  JESS  to  the  players  falls  on  the 
controllers.  In  effect,  the  controllers  are  the  filter  and 
interface  mechanisms  between  the  computer  and  the  players  in 
the  REDCOM  exercise.  The  primary  job  of  the  controller  is 
to  make  the  JESS  as  transparent  as  possible  to  the  players. 
A  good  example  of  the  type  of  role  playing  which  imust  be 
performed  can  be  illustrated  by  a  brief  discussion  of  the 
types  of  problems  which  may  arise  in  movement  of  units. 

The  JESS  represents  terrain  using  a  hex  system.  The 
distance  represented  from  the  center  of  one  hex  to  the 
center  of  adjacent  hexes  is  3  kilometers.  All  roads, 
rivers,  and  similar  terrain  features  are  imposed  on  this  hex 
terrain  model  using  straight  lines  which  run  on  the 
perimeter  of  the  hex  or  from  the  hex  centers  to  the  sides  of 
the  hex.  Because  the  JESS  uses  a  hex  system  for  terrain 
representation,  the  exact  location  of  roads,  etc.  in  JESS 
does  not  match  the  coordinates  which  the  units  have  for  such 
features  on  their  maps. 

This  hex  representation  system  creates  situations  such  as 
the  following.  A  division  commander  may  order  a  brigade 
commander  to  locate  a  battalion  at  a  specific  set  of 
coordinates  which  would  place  the  battalion  on  the  north 
side  of  a  terrain  feature  such  as  a  river.  If  the 
controller  moves  the  unit  to  those  specific  coordinates, 
JESS  may  locate  the  battalion  on  the  south  side  of  the  river 
because  of  the  location  of  the  river  as  represented  in  the 
hex  system.  It  is  the  controller's  responsibility  to 
recognize  that  the  commander  wants  the  battalion  on  the 
north  side  of  the  river.  Furthermore,  in  addition  to  making 
the  appropriate  correction  in  the  location  of  the  unit  in 
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JESS,  the  controller  must  also  remember  to  make  a 
translation  in  coordinates  when  reporting  the  location  of 
the  unit  back  to  the  division  commander,  G-3,  or  other 
appropriate  personnel. 

The  Joint  Theater  Level  simulation  (JTLS)  System 

A  second  simulation  system  which  may  be  incorporated  in 
future  REDCOM  exercises  is  JTLS.  This  simulation  is  very 
similar  in  structure  to  JESS  but  is  designed  for  modeling  at 
the  theater  level.  One  aspect  of  Bold  Venture  87  is  an 
examination  of  the  use  of  JTLS  as  a  means  of  developing  the 
situation  for  the  starting  point  of  the  exercise  using  JESS. 
That  is,  data  relevant  to  the  scenario  used  in  JESS  will  be 
loaded  into  JTLS  and  that  model  will  be  run  for  a  period  of 
several  weeks.  In  the  future,  players  from  the  Army  and  Air 
Force  units  participating  in  the  JESS-based  exercise  would 
man  the  JTLs  workstations  and  be  actively  involved  in  the' 
evolving  combat  situation.  The  involvement  in  JTLS  would 
provide  a  realistic  background  for  the  start  of  the  JESS 
based  exercise  rather  than  the  "cold  start"  which  currently 
occurs  in  typical  CPXs. 

At  the  current  time,  JTLS  does  not  interface  directly 
with  the  JESS.  Plans  exist  for  developing  software  to  allow 
such  an  interface,  but  the  completion  date  for  this  work  is 
unknown.  Also,  it  should  be  noted  that  for  the  Bold  Venture 
87  test  of  JTLS  for  situation  development  for  JESS,  no 
players  from  the  units  participating  in  the  JESS  exercise 
are  actively  involved  in  the  JTLS  play.  REDCOM  is  providing 
all  of  the  controllers  used  in  the  JTLS  run  for  Bold  Venture 
87. 


Senior  Controller  Element 

Perhaps  the  most  critical  component  in  the  REDCOM 
exercise  structure  is  the  senior  control  element.  This 
element  is  actually  composed  of  a  number  individuals 
performing  four  different  functions.  The  deputy  commander 
of  REDCOM  plays  the  role  of  assistant  director  of  the  REDCOM 
exercise  and  functions  as  the  primary  interface  between 
REDCOM  and  the  Army  Corps  Commander  and  other  exercise 
players.  In  this  sense,  he  plays  the  role  of  the  senior 
controller  for  the  entire  exercise.  As  assistant  director 
of  the  exercise,  he  attends  all  major  briefings  of  the 
REDCOM  controllers  as  well  as  all  major  briefings  held  by 
the  corps  commander  and  other  exercise  players.  The 
combined  player  and  controller  perspectives  provide  this 
individual  with  the  knowledge  required  to  direct  changes 
which  may  be  needed  to  obtain  the  desired  exercise 
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objectives.  For  example,  it  would  be  this  individual  who 
would  direct  the  OPFOR  controllers  to  take  a  particular 
course  of  action  designed  to  present  a  specific  battlefield 
situation  for  the  Army  Corps  Commander  and/or  the  Air  Force 
or  Navy  commander. 

The  assistant  director  of  the  REDCOM  exercise  is  also  the 
individual  who  currently  provides  feedback  to  the  corps 
commander  and  his  staff.  The  feedback  is  delivered  during  a 
"hot  wash”  following  the  completion  of  the  exercise.  This 
"hot  wash"  is  an  executive  session  conducted  at  the  general 
officer  level.  The  actions  taken  by  the  assistant  director 
of  the  REDCOM  exercise  provide  him  with  the  unique 
perspective  required  to  conduct  this  executive  level 
feedback  session. 

The  second  function  of  the  senior  controller  element  in 
the  REDCOM  exercises  is  performed  by  the  chief  of  the  REDCOM 
J-3  exercise  control  branch  (J-3  -  EC) .  The  colonel  who 
performs  this  function  has  the  responsibility  for  keeping 
the  exercise  going.  He  is  in  charge  of  all  controllers  for 
the  exercise  and  is  the  primary  trouble-shooter  for  the 
exercise.  Prior  to  the  exercise,  he  oversees  the  selection 
and  assignment  of  REDCOM  exercise  controllers  and  the 
development  and  implementation  of  training  for  controllers 
assigned  to  the  exercise  by  agencies  outside  of  REDCOM. 
Once  the  exercise  begins,  he  settles  questions  concerning 
controller  decisions,  handles  problems  with  exercise  support 
logistics,  and  solves  other  problems  which  may  potentially 
interfere  with  successful  completion  of  the  exercise. 

A  third  component  of  the  senior  controller  element  in  the 
REDCOM  exercise  is  represented  by  the  officers  manning  the 
ground  and  air  master  control  workstations  for  the  Blue 
Force.  These  individuals  represent  the  senior  controllers 
for  the  JESS.  The  workstations  manned  by  these  officers 
have  the  capability  to  make  adjustments  to  the  JESS  such  as 
movement  and  resupply  of  all  of  the  Blue  Force  elements 
included  in  the  model.  These  officers  use  the  master 
control  workstations  to  correct  problems  in  the  JESS  that 
might  produce  unrealistic  effects  in  the  exercise.  For 
example,  the  current  version  of  JESS  will  not  allow  two 
combat  effective,  opposing  units  to  occupy  the  same  hex. 
Furthermore,  iit  does  not  make  any  difference  whether  the 
units  vary  considerably  in  terms  of  size  and  composition. 
Until  one  of  the  units  has  been  attritted  below  a  certain 
threshold  value,  the  second  unit  can  not  enter  the  hex 
previously  occupied  by  the  first  unit.  An  instance  in  which 
this  may  produce  unrealistic  effects  is  when  a  small  force, 
perhaps  a  dismounted  infantry  company  or  platoon  occupies  a 
choke-point  and  is  being  opposed  by  an  armor  brigade  or 
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division  on  a  high-speed  avenue  of  approach.  In  reality, 
the  platoon  or  company  would  be  quickly  overrun  if  it  did 
not  have  an  extensive  network  of  obstacles  and/or  effective 
antitank  weapons.  The  nature  of  the  model  used  in  JESS, 
might  allow  the  company  or  platoon  to  delay  the  division  for 
an  inordinate  amount  of  time.  The  individuals  mannina  the 
master  control  workstations  can  take  appropriate  actions  to 
correct  this  situation  before  it  creates  unrealistic  effects 
in  the  exercise. 

The  general  guidelines  followed  by  the  officers  manning 
the  master  control  workstations  call  for  minimum 
interference  in  the  free-play  nature  of  the  exercise.  As 
illustrated  by  the  example  above,  their  role  is  not  to 
affect  the  outcome  of  the  exercise,  rather,  it  is  to  avoid 
or  correct  any  potentially  unrealistic  effects  created  by 
peculiarities  of  the  JESS.  These  officers  provide  feedback 
to  JPL,  the  developer  of  JESS,  and  to  appropriate 
authorities  in  REDCOM  concerning  the  problems  they  observer 
and  any  potential  solutions  they  have  evolved  for  correcting 
such  problems. 

The  final  component  in  the  senior  control  element  of  the 
exercise  is  represented  by  the  civilian  technicians  who  man 
the  master  technical  workstation.  These  individuals  work 
for  JPL  and  provide  the  technical  control  over  the  JESS. 
The  primary  function  of  this  group  of  individuals  is  to 
prevent  and/or  guickly  solve  any  technical  problems  with 
JESS  which  would  cause  the  computer  system  to  "crash." 

The  QpFop  controllers 

The  final  major  component  in  the  REDCOM  exercise 
structure  illustrated  in  Figure  C-l  is  the  OPFOR  controller 
element.  This  element  consists  of  approximately  50-60 
players  commanded  by  a  colonel.  The  OPFOR  controllers  are 
trSined  in  red  doctrine  and  man  the  OPFOR  air  and  ground 
workstations.  The  OPFOR  controllers  work  closely  with  the 
REDCOM  controller  element  to  create  the  appropriate 
battlefield  context.  However,  it  should  be  noted  that  much 
of  the  OPFOR  play  in  the  REDCOM  exercise  represents  the  free 
play  efforts  of  the  senior  OPFOR  controller. 

Exercise  Process 

The  coordination  of  all  of  the  exercise  components  and 
the  thousands  of  players  and  controllers  participating  in  a 
REDCOM  exercise  is  a  major  undertaking  requiring  detailed 
planning.  Various  elements  within  REDCOM  are  involved  in 
the  planning  and  execution  of  each  joint  exercise.  The 
REDCOM  J-3  element  develops  a  five  year  exercise  plan  which 
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identifies  the  major  iexercises  to  be  developed  and 
conducted  by  REDCOM.  Approximately  one  year  prior  to  the 
scheduled  date  for  an  exercise,  project  officers  (LTCs)  are 
assigned  to  oversee  the  development  of  the  exercise.  The 
officers  work  with  the  REDCOM  command  element  and  elements 
in  the  services  participating  in  the  exercise  to  establish 
the  exercise  objectives.  After  objectives  have  been 
established,  the  project  officers  focus  their  attention  on 
arranging  the  participation  of  appropriate  players  from  each 
service  and  other  exercise  logistics. 

The  REDCOM  J-2  element  in  conjunction  with  the  REDCOM  J-3 
element  works  to  produce  the  overall  exercise  scenario. 
This  scenario  will  be  presented  to  the  exercise  players  in 
the  form  of  a  Theater  level  operations  plan.  The  scenario 
also  drives  the  development  of  the  OPFOR  operations  plan  and 
the  development  of  the  database  to  be  loaded  into  the  JESS. 

The  theater  level  operations  plan  is  delivered  to  the 
senior  commander  of  each  of  the  player  elements  from  the 
participating  services  several  months  prior  to  the  actual 
exercise.  Each  service  element  then  prepares  operations 
plans  for  their  organi-zation.  Members  from  the  REDCOM 
exercise  control  branch  in  the  REDCOM  J-3  element  work  with 
the  exercise  participants  in  the  development  and  review  of 
the  initial  operations  plans  to  be  followed  by  the  players 
in  the  exercise. 

As  noted  earlier,  the  current  REDCOM  plans  call  for  the 
use  of  JTLS  to  develop  the  situation  prior  to  the  actual 
start  of  the  exercise.  This  would  entail  the  involvement  of 
a  small  number  of  players  fron.  each  of  the  commanding 
organizational  elements  of  each  .  rvice  participating  in  the 
exercise.  The  JTLS  play  would  take  place  on  a  fairly 
continuous  basis  for  two  to  three  months  prior  to  the 
participation  of  the  entire  group  of  players  in  the  JESS 
exercise. 

During  the  months  prior  to  the  exercise,  REDCOM  also 
assembles  the  appropriate  controller  teams  for  the  exercise. 
The  majority  of  the  controllers  will  come  from  the  player 
elements  in  the  exercise  and  from  various  organizations  such 
as  TRADOC  in  the  Army.  The  controllers  will  be  assigned  to 
REDCOM  for  temporary  duty  for  a  period  of  two  to  three 
weeks.  During  the  week  prior  to  the  exercise,  the 
controllers  will  be  trained  to  use  JESS  and  to  carry  out  the 
role  playing  functions  of  exercise  controllers.  The 
typical  REDCOM  exercise  requires  several  hundred  controllers 
to  operate  the  various  workstations  24  hours  a  day  during 
the  exercise. 
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Ultimately,  the  database  developed  through  the  JTLS  play 
will  provide  the  starting  point  for  the  JESS.  Once  the 
controllers  have  been  trained  and  all  players  have  been 
deployed  to  their  appropriate  locations,  the  exercise  will 
begin.  The  exercise  begins  when  the  JESS  is  initialized  and 
the  players  receive  their  respective  operations  plans  from 
their  commanders.  The  JESS  will  run  for  approximately  three 
to  four  days.  During  this  time,  the  exercise  is  in  progress 
continuously . 

The  exercise  progresses  with  JESS  playing  the  role  of  an 
automated  battle  board.  The  corps  and  division  commanders 
and  staff  function  just  as  they  would  in  an  actual  combat 
situation,  receiving  information  and  developing  and  giving 
directives  to  their  maneuver,  fire  support,  intelligence, 
and  support  elements.  At  the  brigade  level,  the  commanders 
and  staff  interface  with  the  JESS  and  provide  appropriate 
feedback  concerning  the  progress  of  their  maneuver 
battalions  back  to  division  and  corps  level. 

At  the  completion  of  exercise  play  using  the  JESS,  the 
assistant  exercise  director  conducts  his  "hot  wash"  feedback 
session  with  the  corps  commander  and  selected  staff  members. 
Following  the  exercise,  the  player  units  complete  an  after 
action  report  which  is  forwarded  to  REDCOM. 

As  described  above,  it  should  be  apparent  that  the  joint 
exercises  conducted  by  REDCOM  are  extremely  complex.  The 
exercises  include  thousands  of  players  and  hundreds  of 
controllers.  The  model  used  as  the  exercise  driver  (JESS) 
has  limited  automatic  data  recording  capabilities  at  the 
current  time  and  the  number  of  activities  and  players 
occurring  simultaneously  create  a  necessity  for  development 
of  a  performance  measurement  system  with  a  specific  focus. 
It  is  the  focus  and  structure  of  the  measurement  system 
which  will  impose  some  degree  of  order  and  understanding 
amid  the  "fog  of  war"  which  exists  in  large  scale  combat 
exercises. 


C-ll 


